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POftCWORD 



l)(K>k, uii(l(*rstinMlin<4 is sped il oiir can peer into 
tlu' hook's intriit and rrro^ni/r our s own pliur in 
tluit intc^nt. Scnsibh, a potential icailn* can jndl^c 
whrtlirr hook's cnnipanionslnp will snit liis own 
( iiriositi<'S and ( <)nipulsi«)ns or w lirtlirv lie will sa\ r 
tinu' and stirss 1)V s<<'kini^ otliri. nioir rotnpatihlr 
litoratnrc. 

Tins lHK)k was written for a particular auilii^ntr: 
those athninistrators aad tcwhers on-th<*-joh in Ainrri- 
ran sdMH)ls— tlu» practition<Ts of (*dncation, l<Hatrd 
primarily in elementary scIi(K)1s. 

So far into the sixtie s, the sdiools have Ix'en eanjuht 
np in a ferment ni dexelopnients and ideas whieh 
appear to promise* rather sweeping changes lor tin 
seh(K)ls. Onlv a lew of these developments ha\e lu'en 
svstematically tested in sdHK)l situations, and tlu' pro- 
fessional literature is filled with proi>osals for <'\p<M-i- 
mentatioii and researeh studies designed to support 
or discredit the changes on scientific gromids. lU tlu' 
time that innovations have reached into the schools, 
it has already In^come the practitioner's job to «»\ aln- 
ate tliein if Ik* is going to defend his jndgnuMits and 
decisions. 

But those wlio call for research studies in tlu' 
scluM)ls to acxH)mpany the changes which are confront- 
ing administrators and teachers havi* given notahl\ 
scant heed to preparing materials for practitioners 
which might help ready them for the research tasks 
w^hich are impliixl. 9uite a gap is left Ix'twi^en the 
need for research in the scluxils and workable ap- 
proaches to acttnnplishing that research in reality. 

P' rhaps this lKK)k can at least partially help to fill 
that nup. If \on arc an educational practitioner in a 
sth<H>l svsteni, or if yon are a practitioner in training 
in a college program, or if you are a college professoi* 
working with practitioners, this \xyok may ha\ e direct 
ideas for vou. 

Fred P, Barnes 
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RESEARCH FOR 
THE PRACTITIONER 
IN EDUCATION 



jHfllH Within ihv trachin^ profi^ssioii. and (Mitsidc oi it. t(M). tlu* talk 
^^^H;- alKHit rdiication is Havorrd with terms that wouhl havr smnrd 
^^^^H^ forrign a scant drc-adi* a^o. Modern niathnnatics. tiMcliin^ nia- 
1 chines, electronir hnipiai^e lalM)rat()ries. team teaching, the 
T nonj'raded primary-such is the current \<Kahnlary of echication, 

a vocabulary \xnu out of the (juest to find Ix'tter ways of teaching. 

In the midst of all this ferment, principals and teachers often find 
themselves in a difficult S|)ot. How workable are the proposals for innova- 
tion? I>) the new devices and the new patterns of teachmj; produce better 
results than somo of our time-honored pnK'cdiues? Such (|uestions cannot 
Ih? thrust aside; the practitioner needs ai.swers. He needs answers for 
himself in making decisions alKnit instruction, and he net*ds answers for 
use with parents and other laymen suddenly conversant with tlie termi- 
nology and concepts of educational innovation. Knt when the practitioner 
seeks the answers he needs, when he turns to the professional researchers 
and students for tested evidence, he is often confronted with nothing more 
convincing than research findings of "no significant difference" Ix^tween 
the results of traditional teaching and new ways of teaching. 

The practitioner finds "no significant difference" almost everywhere he 
turns. For example, the Review of Educational Research for April 1962 
reviewed research on technological media completed over the preceding 
six years. The over-all verdict al)out instructional television was "no sig- 
nificant differences" when televised instnfction was compared with ordinary 
classroom instruction. In regard to self-teaching devices and programed 
materials (teaching machiiies), the most popular finding again was "no 
significant differences" compared with ordinary instruction. Language 
laboratories ne out somewhat better, chiefly because the amount of 
research being done was so sparse, and the review ended with the observa- 
tion that much more controlled research is vitally needed. 

It would be a mistake to interpret the "no significant difference** 
findings as either a validation or a condemnation of certain technologies 
applied to classroom instruction. Rather, the findings may indicate that 
the media have been used unimaginatively or that the research has been 
guided by the wrong questions. Nevertheless, repetitive research results 
of this sort tend to build up the suspicion that neither television nor pro- 
graming, for example, has pbwed up evidence to indicate that it is a 
vehicle on which teachers might ride to newer and higher levels in teach- 
ing—at least not yet. 

Another illustration of the lack of convincing research results is found 
in the area of school organization. The self-contained elementary school 
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classnM)m. lu'U rog<'iU'oiisl\ iironiMMl. lias ncnitly vouw in Un rr-r\ainiiia- 
tioii. in |)arl iM ransc o\ n uvwi A iiilnvsl in (h pai Inu nlali/alitMi aiul al)iliU 
(X. V. Z) gn)ii|)ing. Again. piind|)als and U adu rs an' fac<d willi ihr 
"wo siirnifirant difh rrnt |)n//l<'. I)i |)aitnu nlali/<'(l m lu nu s likr ihv Dual 
hoirnss plan ami ihr Aniiilon SrluH)l d\r still Iraguis awav from an 
impartial donionstration that tlu v air arlnrvinu a sivinifirant Jilhrrnrr 
in i liilihrn's K arning as roniparrd witli onlinary organizational ])att<'rns. 
In rdati<)n io irroiiping. tlir NKA's |)nl)liration. Thr Ptinnpals Lo( k at the 
Schoolw^ rcAiahd a nunlad tr<iul. just siiuc 1957. a\va\ from lutrro- 
gcnrons and toward ability grou|)ing in Auu riran sriiools, Tlii . trrnd nmst 
rrttcvi a lialo movi uu nt since rcsrarrli studit's omv ihv past t\vrntv-fi\<' 
ye ars hivv producvd oul\ ind«'t«Tminant findings wliirli have rlrarlv falK n 
sliort oi iK inonstrating an\ allrgrd su|MTiorit\ of gron|)ing arrording to 
ability. 

Some of ihv standard rrf<M« nrrs on c K inrntary srlMM)l ( duration pr<'- 
snit similar rrsrarrli findings and tlu n liaslni to tlu' gnicrali/ation that 
aduiinistratiNr srluMUrs— for c-xamph'. tlu' grouping of |)upils and th'^ or- 
gani/ati<)n o\ subjt t t mattt r-i aunot Ih' r\p«H trd to solv<' trulv im|iortaut 
instrnrtional probh uis anwvay. Of conrsr. this is siinpK a nrat \)Wir of 
rationalization. It arroni|)liNhrs nothing mor<' than to dismiss thr principal 
from responsibility and push tlu* onus for instructional improv< iunit batk 
onto the te ache rs w hvrv it can svrw its customarv di\ isi\<' function, \lor<' 
im|)ortaut. this sid* stepping of the cjuestion also sid<'-st<'ps cogc iit ideas 
about what am be «»\pert<*d to soke im|>ortant instructional probliMus, 

Principals and teachers t annot avoid div isions on instructional prob- 
lems. Neither can the\ rationalize gaps in their knowhdge by lu'coming 
members of the uot-so-exclusi\<* club of "no significant differences/* They 
are. after all. <Migag<Hl in a profession which deals with intellection as its 
stcKk in trade. Small wontler. then, that as purveyors of knowledge, thev 
are expected to appl> the fruits of knowledge to the conduct of their 
specialtN , This is the «'xp«ctation gcMierally held by the public, and it is 
lK)tli reasonable and eouiplimeutary, Kor that matter, educators them- 
selves abjure lack of knowledge, particularly lack of knowledge alniut 
their own bailiwick. 

Quite understandably, then, principals and teachers l(K)k for wavs to 
explain and make sensible their own instructional practices, anv ostensible 
lack of knowledge is highly imct)n\ incing to eNoryone concerned. The 
need for knowhulgi* l>oconu*s particularly acute when a scIuk)1 is involved 
in one of the "new" practices such as televised teaching, programed in- 
struction, departmontali/.ation, or ability grouping. In such instances. InUh 
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iIm |niNn ;iimI IIm' |>r44 IiImnmt r%|M*t iuIK 4Ii\nmi% Un vskU'iHt* i4 
i*N€*«*tl%i*tN*%%. Mi iIm' n*k*\.iiil kiw»iAlf*tl|{r i% (k*Mi|l4ir\ In V4\ iIm* li*4%l Hcil 
ilfl^KifNi^ liiti%l ^m^fUwtf* Im' iiM^kv iiiitl It 1% iiltrti ImhI Io tell wlN-tlw*r miiim* 

%|M*« ihl' |H4ClNl' ti«lt MiIh 4 h IwMiI \I4 llltrllli;i-tM1- iW \4lc*%IIMII%lll|l 

\\ lM*tlM'r lln-ir |>r4«-lMr% lM\r IIn* tililliT iA IIm* in'H ^rr iIm* (tiftil<irl4l4i* 
.i%%iir.iiMr iIm I Msl<Nii.ir\ . il i% 4 i lMr44 Irri^lM ^4 AiiM rM 4ii h IwhiI% Io 
Im' MNi%l4iifl\ W4M liMii; Im iAa\% Io Mii|Ko\€* II iIm' r4i|iii%il€' kiMmk*tlur 
t jii'l Im* IimiikI III rrM*4r( li r\Mk*iMfl- hlktl Hilli iwi ^ii^iiiIn 4iil iliflt*ri imi*%. ' 

|>r4€ lllNNMr% UmA i Im'^ImTI- l<lf 4ll%Ht-|% IaIim Ii Villi M*«'lll %all%l44i« IT \ lo 

lh<*tiiM'l\f*% uimI Im lln^r |Nil»lk- 

Wluil tfri* iIm* tfNi*riMli\r %iMir4t*^ lo wlmli |wftiM'i|MK 4imI li*iM-lM*r% 
tiM\ liirii liir iIm* ftiilofiiMlMNi IIm*\ imi^iI in iikiLiiift iiiUriMliiMul <liti%iiNi^^ 

|lllM*«i INI 4 Mirt III ItNlthHHIIII. llMff %<1*ltl III Ih* %k\ 4|llinMC-|M'% Itl IIm* <|IN*%I 

tm iM*m' kii(mk*<l|tt* iir lor \4lMbl^iii M iiiliirtiMl^Mi 4lrfiMl\ kmmii T1m*m* 
iip|ir«iM(*lii*% r4ii|ti* tfknift 4 (iNitliNMini Initii iIm* 4tilliiirltiiri4ii In iIm* <*«|«i4li 
liiribit. fnnii cli*|MfMk-mv ii|tii«i iIm* IriMlHkiiMl kiMmhilfti* <if *'t*% m ImmI% 
lo n4Mll%r ftiMliiMiMk*iMv 4imI ««*N*ri*li4iMt* ill 4-fi*«litiK 4rMl tt*%litift kk*iift 





WMl twyto^ kllMM. Tin* muM cx m %riiti' H l Mid llir nicnI «^lclrK 
iMrd jatmr lo kiuk lew Mrmt% vt wliAt <lii In llir «4iiv4i In "wImH 

Mllr rftirl In ftml r«liiiiMilr% Im «^ImiI wr air ckntig UV Iirflti mimI 



md oiir M'linli for kiMmlcHlgt* witli tU* ia%Miin|>tNNK ii( c^tHiimtMi |)4rlatK*f. 
Kor fumfilf; tnlifiHt iiMittif c^iiiM»t« wliat fcMiiicl in tfxtlHKik»; xiNinticr 
•liMlrtiCt tlMNikl tfml caiiiMit li*tfrii IIm* tliiii);» cikkf »tnik iits: 

c|iik-k ftUrtrrt ami liard wcHirr» will ksarii iiM»ri* jikI U%U f Ut sc*Inn»I tluin 
latr ItlfMmMTft ^ml r<*lM*U. iiMiittfl tliMiplim* %% imk* i>f IIm* olii(H*ti%t^ t>f 
trjcliinfr. ||t*rr is tU^ 4iitlM»rit\ ci( iIm* tiM|<»ritv. 

Tliu 4|)|>rcMc*|i Id tiM* wiktih kmrn ixluf. with iU tffk*M*tKy ;iiul 
•lirtHiiM^%. w«Milil In- ti*(NifNiik'tfl ; hI |>r€*lrr4hM> if wr i^m\i\ imiIv In* c-imi- 
Iklrfit tlwl wImI t'\tf>lMHl> Vmm% t% trm*. It«it mkIi %% imH IIk- CiiMv It 
may mil luivr Inn^ tnN> to U^in with 4ik1. in 4ny c vrnt. kiMmknl^f iUn*% 
not ftt4tHl Btill. KtHtfit 4tK4iii-f% Ml tiN* %4rMNi\ (itidft of kiMmknlftt* liavt* 
iiMcli' oliMilHr nim-h of tlit* Milifix-t ni4ttt>r c^iffit4iiii>d in tin* triLtlNNikx; 
ps>cliolof{k*al n^M-aith %UH\k^ hiiitt-tl tluit %t*ry tnull chililrni C4n 

Inrni and an* li»4niiiiK \vt\ U% thingft; kitt* Mciofnm 4ImI rvM% niay in 
timr MiqMiu fast ttartrm and lutrd workcmi; and im^ntal disc^iplim* W4ft 
ditcmlHrd on many fnmtt ic^mil %T4rB af{o. 

Taking tlir rmy way in tlir irart4i for knowlrdgr and drpt^nlinK 
upon tmr favorMr aMNimfitioiw r%mtiuilly tirromnt a wtrkmH hindrance* in 
a worU of r«pandlng kirai. It liitclin ih to tin* past. And tmcr hitc'iMHi. 
thrtr it no way oiit. hrcaiMr tlir trrdi of frr«h tliooght arr imM to Im- 
found in what r\rn lvMK alrrady knows. 

Wkal mAmMm My. TIm* nr«t untrvr of knowlrdf(t* is ako tmu* of 
tlir favorHr fmintt ol knowknlfei* ahfMit sc liool iiracikti. Ttir litrratnn- of 
rduc-atkm it IHrd with tlir conc-kitiom. hbndislimrntt. «nd analyirt of 
t^hicalkmal **k-adm/ TIm* iittnit iif this litrratiirr it timpir and dirrci 
niouf{h: if primip«lt ami trarlirrv will rrad or lM*ar tlir kk*4t of ootttaml- 
in% irholan in thr trid. thr\ will In* |Nrrparrd to trantfrr tliotr kk-at to 
thr |irac-tkal ptohlmM of Irm-liinR rliiklrm. Wluit an aiitliority tayt it 
accf plrd at valkl slnipK hraiiiM* an aiitliorU> takl it. 

Thk a%'miir lo vrorkidilr kmiwknlf{r. wliilr iMnctkinrd b% thr acadmiic* 
wurM. it Mrd with iniMMprftrd pitfalk for liotli tlir practHkmrr and tU* 
aiitlHirMy. (>nr pmlJrm it c^imnHinkatkin. To iitr tlir kiiifpuifr of iiifor- 
matkitt tliror>. tlir nir«»4||r trantmiittrd In tU* trtidiT (tlir aiitliority ) nM% 
lir woHiill> Kurhk^ In tlir rrcvi\i*r (tin- |iraciitkNirr ). Ortaiiily thit 
wat thr ctitr with Jnlm IVwry t Imokt on rdoratkui. Tlir rr«^*i%m of 
hit Wirt— yt c*amr awa> wMh nuin% iiion* and f4iii*iful intrrprrtatkNM u( 
hte idnu than lir rsrt inlrtidrd. In tlir taiiir way. intnitkm of static may 
hr dH h^ t ii ig C^manl's mrssnurt cm tlir Amrrkan lUgli school. In Clonant s 
nnnir. a frw hard*lo-hrMr%r chanfrs arr cwrpkif kilo somr of our trcond- 
ar^' schools. 



To read or hear an authoritative dictum and then attempt to put it 
directly into uie without empirical testing is risky business. It is risky 
in the first place because it involves trafficking in other people's thoughts 
and prestige. It is risky in the second place because it is easy to mis- 
interpret the dictum and be misled into doing the wrong things. And most 
important, it is risky because the chain of tlmight-from the authoritative 
production of ideas to the consumption of those ideas-includes no interim 
se<|uence of challenging and testing. 

All of this is not to imply that other people s ideas have only question- 
able use in the search for knowledge. They have great value if they are 
used as m tivators for one s own ideas and if they are thought of as 
proixMitions which must be tested before they are accepted as true. But 
©•ir very reverence for the authority figure in education ordinarily prevents 
our doubting him even in the absence of first-hand evidence, and it is 
always faster and easier to use an authority's name in lieu of evidence. 

Wh^l nnmtihmt My. Increasingly, a show of being acquainted 
with "what research says" is coming to be a popular mark of unassailable 
sophtetkatkm in the educational worM. Certainly, m affinity for knowl- 
edge derived from research bespeaks a preference fm abfectivity and fact. 
Tlie reports of researchers represent a kind of s— iHai^ inCormatkm unoon- 
taminated b\^ human preference or whim; deciaan^ jv- rea^^hed through 
processef quite indepmdent of tlie frailties of hamm Hi aught. 

When research is sanctified in this way, it ohm --^mH an influence 
which is not very difl^erent from the effect of pronowmmts by authori- 
tic^s. TIh* chief difference resides in tlie fact that tlie MNMwrhers have been 
snlntitntcHl for t\w autliorities. Tlie firactitkmer-tlie i tpmmm%er of research 
-is simply getting tlie word from tlie researcher-tlie ffodmrer of research— 
ratlier tlian from the authority. But this difference m nut suflkrient cause 
to suppose that one source is any antlioritarian than the other. 

The researcher himself, wlien occupied with his work, probably offers 
the best evklence tluit this pirifted view of research is mmscientiik. The 
compHent researclier would never approach the reports of other researchers 
as superhuman %vorki cpiite insubtod fnmi humanity. He would be more 
likely to regard tliem as tentative glimpses into another human beings 
way of thinking, glimpses whkrii shoiikl be carefully critkrised and chal- 
k*nged. ^fmt important, lie wotiM probably recognize that the chief vahie 
of soniefMie else's rc^sc^arclies Ikm in tlie clues thMry may offer for furtlier 
stiidk*s of hit own. lie wiNikl certainly not think of them as the final %vord 
on any Mih^*t in wlik*h lie was interest «d. 

As yrt. we liave a cotisklerable distance to go before %ire devebp the 
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wit to use rciiiearch in the satnt* ^-ientific way that it is prfKliiced. It seems 
that we have fallcMi into a iHHion that research is for export, that it offer^k 
conclusive ' nswers. wln^n we sImmiIcI In* using it to provide chies to oiir 
own search for iiiKlerstaiKling. If we allow tlie fiiuiings of research to 
restrict, ratlier than liberate, oiir ways of thinking ab(Mit educational prob- 
lems, tlien scIkn)I people will losc> IIk* fxHential of personal reward that 
comes from l)oing in%'oK(>d in scientific investigation, and there will be no 
l)enefit to tlie vitality of tlu-ir professional knowledge. 

WiMt pm t %tlimm % ImI. The sensate impressions and conscHjnent 
actions of sclmol people seldom find tlieir way into tlie literature of cnliica- 
tion» yet in some cases tliey represent a vakicHl source of knowledge. There 
are some teacliers wlm possess an nn crr iiig instinct for workable, c^ffective 
educational practices. And tliere are wmme administrators ^atio naturally 
seem to tntf muk ingenious procedures far scboal operation. Thme are the 
iniartive paaailMoncTs wlm intellpctuallx leap tlieir way inii stunningly 
ii^pmivt asHNnplishments witiMNit cfHftr being able to rm* inali/c* or 
VMtelize ««Air they do what tliey do. TImpsc are the people ^tm, in tlie 
la«psage ^ amnantics, **ly by tlie seat of tlieir pants/' Whik* tlieir col- 
leopMes aie {lundering. studying, and Msating. these iiatiinU> Met*m to 
anwe eMk at insights and aciiievenMts tliat the olliers can't (piite 
nuMMige. 

Tlie iatiiitive pracrtitkmer can be fowHl in all professions. A |)artictilar 
physician or attorm*y may exhil>it a kind M class in his practice that sei*ms 
to c*scape most of his colleagues, even tkaMKh tlieir preparation may have 
lieen simibr. It may lie tliat wliatever km profession, the practitioner is 
uniquely situated to iitili/e and perfect astiiitive knowledge because* his 
practk^ offers him an immediate Beld ai Irst and feedback for his ideas; 
this is certainly not the situation of Mh^ dsittered scliolar. 

In education, tlie practitiom*r iMtti hMilt-in radar has lone l^tm an 
intriguing but baflling subject for wmmmmdmn. If only his bahavior and 
shrewd tlioiiglit patterns could be d i m htiii and communicat^tf' But this 
is an #lusive ffaal; neither tlie rrwarrkri in tc«cher and ailBanistrator 
training nor liie astaiitive practitiimer hii— I'M lias vH deniomiirtpd much 
aliikty to c)mI|^ tlie secrets. For (')ianipi«\ mie can consult ^ EnofcUh 
IH*Hki of E^immikmal Re^eareh m tin* natalfiy artful t<*aclier nw llir simpk* 
(|iM*sti(Hi. "Mfbai d<i taknitcnl t<*acliefs flo mi |>rodiice sikIi a waAiptttg effect 
on tlK*ir st Ui i tt w K y" and coim* awa\ wit.i tin* c|Nr«tion still iniav4 and tiM* 
answer Uill lrat;nK*ntary. 

As a MNircc of kiNm'kHige, tlie practitiiNier with an instim't for tbe 
visctTuI is (Jiaiinini;. IniI Im* is imH far refiiov<*d frinn iIn^ aiiiioritarian. 

• 
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\\v is sonn^wliat iiiystk.il evrii to liiiiiself, and lliiis caniiol be ptiblicK 
sliarc^il. WisikHii fn>iii this MMirce, like tin* proiKMiiicenieiils frcwn tlie 
aiitlMirity and llie rt*M»an*lier, );eiM*rally \% taken on faith ratlier tliM on 
inMlerstanding. 

WImI f9m€fMmmm% ilMkt. In tin* continnnm of sources of kno^iMge, 
tlH* firsit vigiHTfHis hreak witii tradition, antliority, and mysticism rwters 
witli tlie man from Misso«iri. TIh* tlionglitfnl skeptic wlio demands In be 
xlM)wn iR-fore be will embrace an idea lias taken the first large stride toward 
malnrc, iiKle|)eiu)ent tbinkiiig. 

Of coiirsi\ tlie persMrnt frcrtliinkrr is a triHiblrscmie gny to get along 
with, especially for the MitmriUrian kind of |>enHiti. Fortnnately, tliere 
15^ a wiw, solid cadre of {iMMctpuls anu teacliers wks M»r\c tlie %'er\' nvfnl 
|>itffpiHM« of irritating tlie ^i|M>iirtits f« varioM r^Mratkmal doctrinr« mtA 
criiicisuis. We liafve built ip a kmg-tiaiip cpiidmtsii»**-sN alxHit sncb fMfir 
liiMe%er. and tei ' to ^immm% tkrni mipirk'Mtff^ or simply ImiArMk 
Htxanse tlM*y V I to rewin umnovtii, even in iIm* face of fowwrnb 
acxrpled educational sancsteNis and riiMU, tlH*\ regarded as liard to 
edwate (ask almost any college pnArsMir). ^et practitkmer wlio 
cuNfQriiitally cpu^tkMis and prc*sses for tiie mmmrxs is iMisy stirring up tlie 
serrw fields of acivpted informatiofi anrf tliefpiflvr is of independent worth 
in tkr advancement of knowledge. PbilMplNn |mmiIi Royce put tbe matter 
this way: "Dintpisi* not doubting; it is oHni the best srrvkre thinking mrr 
can render tlieir age. Ccmdrmn it not; it is oNeti tlie truest piett 

Hie coiiicientioiis skeptic is c*iirioiw abMl ntlH-r i^-opk^s i/e«s. Nr 
is abnost comfMilsive abont this <pic^ti«Mng. He is nwii%ated b\ tbe in- 
te H i ia iiml fun ,{ taking idens apart to i mmim tl' tn. In tlie cmt <4 tbe 
skef^ml ediif itional praHiliMirr, this wm^^ trvntg out a new iiistrMi 
tiMMi iwHliod NT nmtenrf w u li as aliNlii gN«i|ii««( m a progi impcI tr* 
l i — k In l m i (-ntbiisiasHi is^^lnwed to tcs Th» tr«4l vmm br an finylliilii < 
tbMijgh •vtber formal or niivmal es p i wt ntaHiM4 witliin tlie cvmiitrs «f 
our ^ fiwrn After all, mm |irocvdiire. kr it m old, is recoMnenM 
to iia^l p i ipl i lircanse smtoih* lirlieves \w Urn e^ iiletice of its prrirr^ 
ahAlNr for use in certain sitmitkms. Wliat, tkm n UMNre directK k>fpral 
tliM to give tlie procednri' a trial in «nie of tlie Mtiiatkms for wliick it 
wm Mtrndrd'!^ 

The tvdonbtabk\ cinestnniing practHttNM>r malir% a solid contribnticMi in 
tlie snirc*h for kncnvknlge, fust by poimiiig to tlir tmigli sort of c|iM*«lmis 
tliat iNiglit to U» askiHl. Hut of coiiis<v Mmtething else is really netM. 
If itie dcMiNer only foinises kis attimtkNi on dissA^Hiiig otln^ |)eople s Mras, 
bis fNrn original thinking tends to lie mnvckxl <Nit of bis pnrview, and he 
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remains stimulating lint incomplotr 
%tv\y toward scientific tl inking ami hb 
just this much is to drink tlie apeiM 

tuMicr wlio niminates a good deal mm 
is soon(*r or later IxMmd to risk trvaic 
mistakes. He will liecome impatient^ 
edge to ttie mistakes that are u*v\ 
preference for indrpendenci* vvi 
proach to educatim bastnl n|K)n 

Tlie sense of following one's 
tlirotigh rigorous test, is self-fnl 
more, scientific iiM|niry represents^ 
enlarge liis verified knowledge, h m 
hufiian entrrprim Proficiency m 
dures elevates tlie icIkioI practiti 
around him. AiiHiorilati%-e pron 
and iclrat from alt other sources mi 
leads which are prrkaps worthy 

it would he a sertmis mislakrm 
lag and grandiote. Researdi can kt i 
simple, as anyone wants it to be. tm 
tlie better. Far from being an i 
dressed in a white gown, research 
comfortable as a wdl-wom suit 
clear s UV iiicn t on tlw tlie me was 
and it 'fifi^ ratlier neatK to the 

'*Voit have all lieard it repenkdL 
by meaaa of induction and dedb*< 
operatiaM. tliev-, in a tort of sense 
wliicli are called aainral laws, and 
cunning skill of tlMr own, they 
it is imagined by many, that tlie 
by no means cony id with tbaar 
acc|uired hy a sort ad special a{ 
large words, vou wmild think fkm «a 
be coMiMnled difci an tly from tkm «rf 
be f hp^ aned by terms, you wN dbr 
that all tiMe terrible apparatw 
and every hour of your lives.** 



has taken the first ver\ ntvessary 
aigatioii, hnt to lie satisfied with 
mI mi^s tlie meal. 

ive school person, the practi- 
titr embattled state of the schools, 
be own ideas and making his own 
ni bring letl hy second- liaiid knowl- 
lo scIkkiIn m flux. This 
into a twi ledgeil ap- 
'ntatioii asnl research, 
and tbeti %-erifyiiii( it 
rewarding. Furtiier* 
Mirougb which man can 
ii supreme importance to 
aiid reseaerh proce- 
ilie iirtellectami world 
far IcM commanding, 
as tentative 






to sonsH overpower- 
, as amhitims or as 
in buaniity, sn much 
set of iiitant rituak 
it b eminu i s as personal and 
ba)(gy tfouaers. A wry 
Tbomas Henry HutAt\\ 
deMloped here: 
•bat men of science wark 
■r b\ the help of tb^e 
certain a*er thsBf(s, 
out of tbear, by same 
and timories. And 
at ibe common mitid can he 
tbat they bnve to be 
to tlw amft. To bmr all these 
of a amn af Mence must 
bnt if yon will not 
^pMe wrong* and 
• ^ ^1 vouffseb^es every day 
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Gaining ac(|uaintance and familiarity witli Hnxley's *'terrihle appara- 
tus" is an advisNible antidote for the mystery and bunkum whicli surrmmd 
research activities and their products. lndc|H>ndent thinking and ingenious 
testing of ideas--togetlier with appropriate methods and technic(ues for so 
doing— are not difficult and forbidding. If they were, it is doubtful that a 
workable scientific u)ntent could ever lx> achieved in education because 
it is tlie practitioners wIm> are in dircc*t contact with pupils and hence 
through tliem that scientific findings finally reach the classrooni. Indeed. 
priiK^pals and teachers are already using the thoughtful metlMxls and feel- 
ings of science and research "every day and ever)' hour of | their | lives," 
often without knowing tliat tliey do so. 

It is, then, to tlie reflective school person, to the principals and teach- 
ers who wish to find out for themselves, that this book is addressed. In 
the search for knowledge, the practitioner wlio understands and utiliva^s 
the methods of research will find himself aliead of tlie game. Ratlier than 
being helplessly frustrated by reports of **no significant difference" or 
mutely obeitant to the prououncements of autliorities, he will have at bis 
command the tools with which to search independently for the knowledge 
he needs. 
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Research is a way of dealing with ideas. It is iiothiit^ more than 
^^^^^1 this, and it is nothing less. MoM research deals with ideas tffoni 
^^^^^1 the standpoint of one or tlie otlier of two types td cH-ienftiiion 

^1 tewid time. The fir>t t\ jx* of rcsi*arch \ooVs from the pment 
^ Id thr past; it rrprescnts an attempt to cxphiin wliaC has alrrady 
happened. Historical retiearch and surveys are two examples frf research 
that looks iMfawiH ns at where we have liern. Thr nicth(Hl$ iisr^ are chirfly 
descriptive m nature. Tlie second typi* of rcs<'arch n^vcrses tWr uirientation 
toward timr aaid looks from the preM-nt to the future; it frpvr4»ents an 
attempt to intwiiat would luip|)en if . . . Almost all experianri^ research 
in the physiisdl hkilogical and hehaviotal sciences is an exaiH|4r ^ research 
that looks alvad of us, at where we mi|«ht be if, . . The ntetfMk used are 
chiefly infevmtial in nature. 

In both historical and expi*rinM*tital rrsiMrch, the ideas of tlie'sesearclier 
and the ways in which the ideas are haiKlled constitute the reivavch. Tlie 
liistorian attcsnpis as accurately as posMihle to descTiln* what lir thinks has 
liappeiied; the experimental scientist attempts as precisely as |wsibk* to 
test what lie thinks wnuy happen if relevant controlling fac^s were 
deliberately altered. 

This book is chiefly cmcerned with experimental research. The ex- 
perimental research p ro iass utilizes c < att ai n approaches to tlmnght and 
investigation wliich liave been developHl in order to increase the likrli- 
liood that the restilts of the resanadi wM be rek*vant, sensible, and use^d. 
Tlie essential nalare of tlie rearasch paacess is discussed in this chapN^r. 

lUlJIllHiiyi m IMMMMIAL RISIAKN 

Experhnental research has certain funAvneiital charadrristics. The 
ways in whtdi eaim-inmital mearch does nail does not albwipt to lest 
ideas are evident m tlie follifnang discussion al ten of these calaBacteriatics. 

1« Wmmmdk idaos am sMtrktvrf by ^ rwiuiram^iH Hiot Ihvy 
bm taatoMa. Alnwt everyone has an flitasianwi stirring and nnwual i^a 
which soon witlws for lack nt a promising ssay to try it out. Tlie efcrt 
of folowing an inkta or inspiratsan to its kips si cnncliisian is wmthmMie 
only when tlie researcher has a reasonaMe asansMce that dhasatraiSBBi nr 
expevimenlatinii in tlie na ln wa l w«sM cav p»n%ide tiNf vmS±sA mkmmuiikmm. 

of thr moat rigaaans laals af a thr !r> is Id ask liw adeq n ai ik m ex^pkmm 
phenainena in the natnual n u i M ^Mnat ol ns have liecome s i'm s— n iii f d to 
that iatal dualism, "Mnvhe it mH wmk m thran. Imt it mam t waak in 



prat-lKc/ Tlir rcscarrhfi is likely to insist tliat a tliroix wliicli (aainot hr 
\crihf(l tliron^li inipinial irsiaicli ami nnIhcIi won't work in piat^ur was 
prolKihly an inconiplrtr tlirory to Im ^in witli. llv is likeh to n i^ard a 
theory as owv of tin* most iisrful waNs to rxplam piaitical r\riits. If a 
theory really is <li\oref I from thr actual woiM. it is iK»t sriAing its lii^lirst 
function and is nH»st likch onl\ niasc|U(Ta(lin^ us tlirory. 

3. Simplicity in ideas and conceptualizations is the ideal. Direct, 
clear ideas are \ery difficnlt to conir l)\ and maintain. It almost sernis 
that tl K* more ''dncation some |h*o|)I(* a(M|nii'(\ thr more complicated and 
cluttered their nieiital inachinerx lH»c«)njes. Tins slate <rf aflaiis is siarceU 
an asM»t when it i^niM^s to solN ing proMems. clri^ant idea in r(\srarch 
is fretpittitl) clear and oI>\kmis that others are lu ard to ask plainti\elv. 
"Why didn't I think of that?" And it is the simple, clear idea which \vr\ 
often is the richest unci dec'iK'st, Who has not Imh u inipressi^l with the 
degant formula of Eini^tein's theory of relativity, l\ \t( - f lanlnKlit d in 
this seemingly simple formula are the iwvk ideas of space, mass, time, 
motkm. and {^\ it:'tion that laid the foundation for atonik- fission. 

4. Reteorch s#ts out to test, not to prove. "Proof" is a loijical 
ini|)ossil>ility and (juite U^yoiKl the realm of research, T\w res<%ircher M-eks 
answers to ({uesti >ns through the application of scicntilk* pnKe<lures. lir 
docs not think of himself us a su|x*rmortal out to discmer and pro\ nlti> 
mate truth. Instead, he is liki'ly to n'gard himself rather lunnhh as a 
tester of ideas which may prmick* clues to the solution of problems. TIk^ 
phrase "research pro\cs" i>osstbly applies to nothint; more gk>rifi(*d than 
toothpaste and aspirin advertisi^nients on television. 

5. The concept of "failure" is on orchoic inlvrferenca in research 
octivities. In a very real and useful sense, it is iM)t |K)ssil)le to fail in 
research. Tlie expwted norm for research in all of the sc*iem*es is *iM)n- 
significance/' Tins means that research projects wliii*li result in "fOpM^ieant** 
findings are by far in tlie minority. In man\ c*asr«. tlien* ts hhwv to he 
learned from tlie reesons why socne h\|inthests faiU to r^eh significance'* 
than might have been leaum^ from a suppondk sni^-ssfnl ex|)mmeiit. 
Tlie intrtMien of tlic failitir concept hUo reaeaerli b» aMiscientific; it €*n- 
Climbers IIk researcher wiHi implied pmk ferllH|pi and with a morml judg- 
ment whkih is quMe foreign to resewch ob)ectt\«. 

4. m» potew W ol volue of o i hb i l Ii pMpsf is Arectty related Id 
1^ coQMy of the qiintions athmd. Remudk always starts from a 
^■rstion or problem of some sort. Thrrr are two ^pmeral kinds of reemns 
im askii>g Mearch ({ttestions: inteUvctiiei rewans, hiMed on the desire to 
kmm^ %imfk far tlie satisfaction of hi n w i ng; end practical leamm, based 



Thr Research PracvMi 



on tlie ilesirc to know for tlie sake of iH'ing able to do something Ix'tter 
or more efficiently. The first kind of (jiiestion is sometimes seen as leading 
to "pure" or "basic" research; the second, to "apphed ' research. 

At times, these two types of rcseiirch I're discussed as if thev were 
mutually exchisi> e and as if one were l>etter tlian the other. But in actual 
practice, research on practical problenis may lead to discover> of basic 
principles, and intellectually nioti\ ated research max yield knowledge that 
has immediate practical us< iuhiess. Whether the motivating purpose of a 
given investigation is primaril; intellectual or primariK practical, the re- 
quirements of good research ^WK-edure are essentiallv the same, 

7. The methods of reseorch are intentionally devised to prevent 
the researcher's deluding himself and others. It is so easv to believe 
that facts have been demonstrated, when in realit) nothing has been dem- 
onstrated at all, that the precai*tions common to research seem downright 
negative. For example, in a statistical study, we work with a "null" (or 
"no difference") hypothesis. If we reject the null hypotliesis at the five 
per cent level, this means that we are willing to be wrong in that decision 
five times out of a hundred replications because of sheer chance or sampling 
error. In a more or less complicated research study, the chances are so 
great that our presumed findings will be confounded and that we might 
be mistaken in our decisions, that we lean over backward to acc*ount for 
possible errors. We try, so far as possible, to make certain that our ques- 
tions and procedures are relevant, reliable, and unbiased. 

Voiues ploy a tegitimate ond impoHant pcNt in iMeorch 
octivities. Not too long ago, research was thought to br co«pkteiv ob- 
jective, dealing only with facts. Because values are subiMtive and con- 
cerned with human preference, it was taken for granted that tbev could 
not be usefully investigated through the methods of research. This may 
have been true at one time, but the behavioral sciences have now worked 
out techniques for identifying and comparing values through altitude 
and opinion scales and the Uke, Ingenious ways have been developed 
to transform what seem to be questions of value into questions of fact. 

Opposition to continuous promotion through the grades, for example, 
may he based on values having to do with "upgrading the standards of 
academic work" or "using the threat of grade failure" as a motivating 
device. Whether grade faikire does or does not result in improved stand- 
ards, or whether it does or does not provide desirable motivation, is a 
question of fact and therefore open to answer by research. In other words, 
the empirical effects of values can be systematically studied and inferences 
drawn concerning the values themselves, 
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Values an* also an important part of rcsrarrli in a quite difTcirht 
context. Tin rrscartluT lias a moral ohlijjation to tlir snbjnts of Ins n* 
search. In (Hliieatkmal research, the teaeher anil administrator lia\e an 
etliical respwiMhility to stiuhnts, parents, anil tlie selnM)l eonnnnnitx. It 
is no inore cit4ensihle to employ experimental deceit in research under- 
takings invokujjr students than it is to limnlwink students in the educa- 
tional prwess. Fn-fjoent use of what is regarded as deceit (for example, 
projective t\|>e tests which nias(juerade as ordinary achievement tests) iua\ 
undermine tin* niK^essary ct)r.fiilence hetween in\est]gator and suhject or 
In^tween teuc<ki*r and student. Students, as well as researchers. ha\e per- 
sonal vidnes winch ari apt to l)otome invoKed in the research pnKrss. 
To assume tlitft these are freely exploitahle is as unreasonahle as for a 
surgeon to approach a healthy man with the recpiest, Pardon me, sir, may 
I make a dvvp incision in the interests of science?" 

Tlie primr etliical principle is Iwsed u|>on the recognition that achie\ e- 
ment of the imestigator s ol)jc»ctives is dependent n|K)n his respect for the 
suhjc>ct's vahn. While the principle is applicahle to the entire field of the 
l)ehavioral sc«ces. it is particularly crucial when the snl)jiH;ts are a cap- 
tive audience mi school children. Failure to res|H»ct the children s values 
is totally incamistent with the purjxises of education in our society. 

9. The NWlhuJs of analysis, of logical deduction ond stotistical 
inference, $hmM fit the limitations inherent in the problem being 
investigofed. H bas become quite fashionable in the world of teaclitng 
today to borrow— m at least strike the stance of borrowing— ideas from the 
"basic disciplines/* In fact, the "interdisciplinary approach" to school con- 
cerns has aloMMl become a panacea for educational ills, despite the fact that 
the exact meamig of the phrase seems to be considerably more than hazv 
to those inteffdiscipknarians who use it. Of course, Ixirrowing from the 
parent fields (who frequently disavow paternity) lias long been a trade- 
mark of the schools and of education. But the cost for so doing has been 
heavy. Too fmpemly, tlH^ schools and stndents of education have bor- 
rowed both the tnetbods jmI the findings of research in the related fields, 
only to discover Hwt neilber the methods nor the findings fit particnlar 
CKlucational |— Mi im 

Ralph Tyler. DaKtor of the Center for Advanced Study in the Be- 
Imvioral Sciem, poisKed m the lack of "fit" at the First Annual Phi Delta 
Kappa SympaMM on Ediecational Research in 1960. "Because of [the] 
difference in iht kinds of <|uestions under study, the behavioral sciences 
not only may fail to provMfe direct answers to the questions of primary 
concern to edmtional mnrch but they may also contribute less in the 
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nature of research findings than they do to the development of conceptuali- 
zations useful in planning and interpreting studies of education and to the 
invention of techniques for designing studies, and for collecting and ana- 
lyzing the data of educational research.**^ 

This is not to imply that the behavioral sciences lack value for educa- 
tion. Rather» it is to point toward the need for discrimination between 
the techniques and methods which fit our limitations and those which 
uncritically do not. For example, education is a purposeful enterprise 
conducted to accomplish certain social and personal ends. Education is 
directed through appropriate objectives which make it goal oriented. The 
basic questions in education tend to be directed toward the determination 
oi objectives; the selection and organization of educational experiences; 
the selection and guidance of students; the kinds of educational personnel 
required; administrative organizations, policies, and procedures; selection 
and use of instructional materials and facilities; and civic support of educa- 
tional institutions and programs. Logical and philosophical analyses can 
be instructive when they meet these particular kinds of limitations. When 
they do not» they tend to vaporize into abstractions. 

Similarly^ it is highly possible for statistical techniques to squeeze 
educational problems into mathematical models which distort the real 
problems of schools. For example, how shall we deal with a small number 
of cases, such as the twenty-five or fewer pupib in the ordinary classroom, 
when traditional statistical tests of hypotheses improve as the sample size 
increases? How can we test, statistically, an obviously non-normal group 
of children when statistical OKxlels tend to be based on the normal dis- 
tribution? How do we interpret research that involves huge groups of 
children? 

These problems, and ones like them, tend to sound quite defeating. 
Fortunately, research designs and analyses which come very close to 
fitting the limitations currently present in actual school problems have 
been worked out, and new ones are being invented every day. 

10. Ihm rMMirch«r courts recognition through the power of hit 
tosfod idoot, not through the ottroctivenest of hit rhetoric. Eventually, 
most investigators find it necessary to shift the focus of their attention 
from analysis of data to written communication of their findings. When 
this occurs, the researcher becomes a writer, but he is still bound by the 
same careful rules intended to avoid self-deception that characterize the 
conduct of research itself. 

Unlike other writers, the scientific author is not free to choose what 
he will include and what he will leave out on the basis of the effect he 
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wishes to create. Tlie basic rule wliicli the investigator must ohserxe is to 
give all the evidence that is relevant to the research (jnestion asked. This 
he must do. whether or not the results are in accord with his own views 

To do a good job. t!ie research writer will tell enough alxint the study 
to enable readers to judge the adecjuacv of its methods, to form an opinion 
of how seriously the findings are to Ik- taken, and to rcfK'at the study with 
other subjects if they wish. Crucial jwints frecjuently included are the 
study design; the method of identifying and treating the variables; the 
nature of the sample; the data collection technicpies; the method of sta- 
tistical analysis; and the specific levels of confidence accepted in deciding 
whether differences are to l>e considered significant. Tliis kind of detailed 
writing lays its claim to distinction in that it invites the reader to make his 
own independent check of the procedures used and conclusions reached. 

Preferably, the research writer will l)e very careful not to claim more 
than his data and analyses will allow. Other writers may dramatically 
clinch an argument by the trick in logic of generalizing from a particular 
case to an entire universe: "Johnny can't read; therefore, most children 
can't read." But the researcher only discredits himself and his work if he 
indulges in overgeneralizations and pre-ordered conclusions. 

TWO WAYS OP RIASONINO 

Students of semantics have fairly well established the fact that our 
ideas, concepts, and thoughts depend greatly upon the language we use. A 
person who speaks a language that has an entirely different structure from 
that of English-such as Japanese. Chinese, or Turkish-does not even 
think the same thoughts as an English-speaking |>erson. Because the re- 
search process deals primarily with the creation and manipulation of ideas, 
the researcher needs some consciousness of semantic determinants and their 
implications for reasoning. 

Logkcrf-vwbol rtcnoning* The thoughts we have depend largely 
upon the words we use. Our words are verbalizations of the ideas they 
represent; as such, they serve as clues to the meanings we seek. Depend- 
ence upon verbal clues means dependence on a highly developed, subde. 
and complicated form of symbolism. Through the centuries, human l>eings 
have agreed to let certain sounds (speech) and certain marks (writing 
or printing) stand for specified happenings and plienomena. The symbolic 
procens is the process through which human beings arbitrarily make certain 
things stand for other things. 

As human beings, we can manufacture. manipul»^e. and assign values 
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to our symboli. For example, we ciiii agree to let X stand for the hoys in 
a school and V for tin* girls. Then we can let the symhol \ stand for all 
the X*s and Y's. S hecomes a symhol of symhols. It is important to note 
that tliere is no mvessary connecticin hetween the symliol and what it 
symholizes. This is tnie of all our words and phrases. The word home 
hears no resemhiance to tlie characteristics oi tlie animal on tlie racetrack, 
the cowpony on the range, or tlie Shetland on the larm. It is a symbol- 
an abitraction-which expresses what is common to the racehorse, the cow- 
pony, and the Shetbnd. ignoring the differences among them. 

The symbolic proceii is made pouihie through the fm^en of abHract- 
lug. This proceu is an indispens4«ble tool with which we select certain 
characteristict of an object or event that suit our purposes and leave out 
other characteristict. Our words, or verbal symbols, are a form of short- 
hand. They are abstractions which permit us to hold problems in place 
while we work on them. They permit us to compare situatkms Ly examining 
likenesses or differences. They also wake it possible to build conccptkms 
which have no direct counterpart in the world around us. In fact, every- 
thing that we know is an abstraction. 

Some of our abstractions are at a higher, more inclusive level than 
others. Highly abstract terms and phrases uhimately serve as classifications 
and generahaations. Terms or phrases at lower levels of abstractkm help 
to nwike these generaUzatiom clearer and more speciffc. For example, the 
generaliaed coooepts of "work," "play," "oommunicate," and "read" are 
at a raiatively high levd of abstraction. They stand for characteristics that 
a multitude of activities have in common and leave out most of the spcdlc 
characteristics of these ac tivities. The abstractiGn "work" does not neces- 
sarily imply rowing a boat, buildhig a garage, or managing a business, yet 
all of theaa activities amy be work. "Pky" might mean anything from 
having a gaaae of checkers to indulging in pun*making. "Communicate" 
could vary from one person s sly wink at aMthar to sending a tele^am, and 
"lend" saight mean an>ikiiig from inliiipiiillng a musical score to receiving 
MoffM code ovev a lelegmpk. 

HwImI ol M alMtnctfcM is Ml wWdMr it is hifk or low levrf, but 
wImUmt y^m a lliaeU DM S cm b» mttmlmffwM} tshlsd lo lower mm 1m 
iIm mm sjiUmi oI iihtiowlilpi; whtllMr kicMy abilract idMs cm be 

iwylilrwSj tiHiit nT * iT'- Nor is thMiralicd think. 

^tdm tvM oractieal Atmklmm. A Denon who has oommimI 

O 

ERIC 



StafitHcol rMMniflf. Tn this |X)iiit wr \u\vr Ihtm ((insidcTinji 
logiciil viThiil appioiuh \n Ihr irsrairh prmrss. nnlHiUUd in hnipiiivir 
s\mlMils anil ahstrartitins. llour\(M . thnr is anotht r lan]L;na]L;r. n)niiM>srd 
of niiithrniatiial snnlnils and abslraititnis. which is known as statistics. 
I ndcr certain conditions, statistical lanmianccan oprcss the same thon^hts 
nion- ccononiically and allow ns to reason in wa\s which the verbal ap 
proacli cannot encompass. 

lM)r example, we max desiriU' a fifth j^rade uronj) of pupils in two 
ways, nsin^ the s\ inhjls and abstractions of verbal lannnane and of statis- 
tical lun^nuge. 




TIic illuitrelion in Figure 2 makes mm* of de.tcriptive siaii9iic9. In moft 
iniUncei, ihii kind of Klalitlics enables ns to \^ r{ ,„j f\^i^i ^| ^ relatlvelv 
higli level of matliemalical aUlractioii. | with tlie very 

weiglil, bulk, and inexactness of words. Afn^^ .t^ ai stati^fics allows 
us to do still other things with o^ir tlioiights wI.m I itln-r k verbal ap- 
proach nor descriptive statistics can encofn|>ass. Tkt-se otiier things are 
associated with statistical tests of liypotlieses aiid statistical inferences. 
This branch is known as lmferenik$l at^UulicM and is based on the mathe- 
matics of probability. 

We know, for example, that any two measurements of phenomena ( test 
scores) will differ to some extent |ust h> cliance or because of errors in 
the measuring instrument. Unless we liave some way to accoimt for the 
operations of cliance or tlie emirs, we liave no means of estimating whether 
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tlie dWrrtMice wc note lH*t%vrrn the two measurnnents is cine to chance 
factors or wlietlier it i.s .significant. ProlKihility tests gi\«* ns a way to 
estimate the intrnsion of chance hahniced against the effect of our predic- 
tions. For this i)nriM)se. we ch(M)se an appropriate statistical test (mathe- 
matieal model) to apply in the analysis of onr data and then consult a 
cwresix)nding table of probabilities, found in inferential statistics Ixioks. 
Such a table allows us to estimate the extent to wbich sheer chance, 
sampling errors, or measurement errors c*ould or could not have produced 
the results we observe. Then we are able to re|X)rt our findings as "sig- 
nificant*' or "not significant** at a siK*cified le%'el of confidence (traditionally, 
at tlie one per cent or tin' five [kt cent level) in judging tbe tenability of 
our research hy|x>thesis. 

Tlie foregoing discussion is a gross description of tbe pnKcdures gen- 
erally eniploycil in following tbe chain of statistical reasoning. Later we 
will return to a more siX'cific consideration of certain statistical tests and 
tec^hnicpies which are esix>c*ially useful for tlie educational practitioner. 

CancvrrMt wm of loficol v rhol <nmI U ^H kHh aH mmmnin§. Statisti- 
cal analysis can never Ik* a snlnititute for thinking. No amount of statistical 
sopbisticatiou can comiKMisate for illogical donclusions. Most frtnpiently, 
illogical conclusions result from errors of logic rather than from errors of 
technique. What is needed is a proper understanding of tlie contributions 
that lx>th tlie logical-verhal and tbe statistical chains of reasoning can make 
to research efforts. Figure 3 |X)rtrays tlie interdependence and similarities 
of the two ways of reasoning. 

Statistical metliods may he regarded as metliods for measuring, de- 
scribing, and analyzing observations which have been made. But while tlie 
chain of statistical reasoning can lead tlie experimenter to greater precision 
in his attempt to quantify his observations, and while it can help to prevent 
him from drawing unwarranted conclusions concerning his data, this is not 
tlie whole job. Rules of evidence for drawing valid conclusions are also 
rc(|iiired. The researcher is still faced with the necessity for logical-verbal 
reasoning to interpret the meaning of his findingv As - mU, th< * ^ » 
constant interplay between the statistical and ''>r »al ci4«4ii. 

reaioning in the research process. 

Sometimes the experimenter can apply onh logical* verbal methods. 
There are many problems and questions related to teaching and school 
management for which a statistical model simply will not suffice. It may 
be that ways to measure and express certain phenomena are not known, or 
that certain behaviors cannot be quantified satisfactorily. In such cases, 
the onl)' available alternative is to proceed toward problem solution on 



21 



Hrwarch for tlw Prurtitionrr in Education 




Fifwr* 3. Th« R«f«orch Process — Two ChcNiis of R«osoning' (Rvad from />^>Wof« up) 



the hasis of logical reasoning. But the experimenter can employ the same 
sort of careful thought patterns common to the more precise statistical forms 
of reasoning. 

SEVf N STEPS IN th| MKAICH FlOCiSS 

lU search always start lestion or a problem of some sort: 

Does al)ility grouping procUii^ (xisitive attitudes toward learning 

than does mixed grouping? Is rcaduig comprehension improved by using 
an individualized approach as opposed to the traditional reading group 
approach? Should sixxrial classes or spcK^ial scIhioIs be organized for gifted 
children? I>o students learn more and better in specialized areas such as 
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that it is pos5s^le to appK w j iii i' ^icesses in staking answers The 
l>roces9es that are used loped in ordc to inereuNC the 

relevance of the infomiatio«« iL4die- 
<fiiestions, of co«irse, are no' lifldbM^io 
prompted by rHiwtions on lihr 
-uces as we kiH»w them or 
IS to test onr iMmclies abm«' w*» 
-night affect one another. 

As the various sciences n^IHgiPTl. tkey have developed effici'^nt 

m^tikiods to communicate ai^J lIis thr inetbods of the research process. 
f<> tacilitate discussion of f ir wcHiorK. it in convenient to organize the 
^s<*arch process into a liri i ot "svyi* Customarily, such organizations 
^mmm to carry the i«iplicati«» tint «^ 4 always should follow step 3 and 
irp 5 always shovU prccwdt I This unfortunate and mechanical 
irtrrpretation of the reseawli pHM^ m «imply the result of confusing the 
t«p(ical form used in talkmf^ akmm lii mmrch with the actual occtirrences 
thmt go on in d&mg it. 

There is nm special nicnt in kmrn^^mUt merely to name or identify a 
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findings imply for tlie original problem? If not significant, how 
might the problem be approached from another direction? 
ST» 7. Rejection or occeptances In a statistical study, we always test 
the null hypothesis to find whether it can l)e rejected m favor of 
the alternative hypothesis. If we are willing to run the risk of being 
wrong in this rejecting, say 5 chances in 100. and our probability 
level is shown by the appropriate table to be .05 or less, then we 
may decide to reject the null hypothesis and accept the akemative 
(or research) hypothesis. But if our probabilit> level is sliown to 
be more than .05. then we cannot reject the null hypothesis and we 
conclude that the results obtained did not indicate a "sipiificant" 
difference between the experimental and control groups. 
In practice, the seven steps of the research process seldom follow one 
another in a fixed order. On the contrary, each step in the process does 
something to perfect the formation of ideas and to promote the location 
and definition of the problem. Each improvement in idea formation and 
problem definition leads to new observations that yield new data which 
aid in judging more accurateU the relevancy of data already at hand The 
elaboration of the hypothesii does not wait until the problem has been 
defined; it ma\ come at any intermediate time. The introduction of any 
new step need not be final; it may be tentative and exploratory, later to 
be withdrawn or revised- In practice, two or more of the steps may tele- 
scope; some of them may be passed over hurriedly, and one or two may 
be chiefly responsible for the conc*wons reached. The way the steps are 
managed depends upon the nature ef tlie problem and upon the sensitive- 
ness and imagination of tlie investi^tor. 

One more idea should be noted about the handling of the research 
process. Frequendv. steps are take* and observations are made that prove 
to be irrelevant to the main point being investigated. False clues are fol- 
lowed; fruitless ideas are entertai^d; superfluous techniques are tried. 
Because tlie solution to the problem is not known, it has to be groped for 
in the dark; lines of inquiry are started that in the end must be given up. 
The logical forms that ch«cterize conclusions readied and accepted can- 
not, therefore, prescribe the way in which answers are sought when a 
condition of doubt and i«T"'y The printed reports of completed 

research projecU present very misli«ding accounts of the ways research 
projects are carried out hefore idem become definite. Seldom, if ever, do 
published reports tell of false starts, wasted hours, personal anguish, or 
personal exhilaration over unexpected successes. Yet all of these experiences 
and emotions are common to research, so long as it is in process. 
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sUndaniBKtl tcits— twit cTeativitx iiu! 

Tlir ^rd tim^ of (|m>stioii wliicli is not rcsranciuiMr dfuls m-ttli n 
ciMice m k-ciami wlikrli involxe-s valin-s, as wdl as ndUrmatmn. "SlMmld 
the state^ icgiilatim* estaNisii a cumpnljiory swstt m hw ikm^ ftiioridatinn of 
drinking wirter to wiprnvi- eliildrc-ti s dental lieahh? ^ » a ({ttestioii of this 
type. TW answer mvolves not only factnal inffmnatmL «neli as the r#r^t 
of fliMrme m tl>e pn-vention «)f dental caries, hwt also pr««rioiK» beliefs %ncli 
as tliose lieU by members of the Christ ian Svw^icv faitfc amd political atti- 
tudes whicJi tend to reject an\ ^emniental iwrcion. Tbere is no scm n- 
tific Mthod^ testing tlie validihf of these sakmrs. OnK iAk^ cunse<inenc<^ 
max be Vrwmm. Vabie judppnents-as they affei-t flH> liehairw of others and 
of tkr researclier liawelf-are a legttiiM^ wworni of i \ m rwarch> 

ers and gwd tecfan^ies arr kwwn ior •htemiig iwd ahMl them 

TW central positiw M vahn* |iij ^ ;i n mts in eAw 4finMl sKuh lies ia the fact 
tl)Jt tliey are tlie ^«Hiali/ed exprevsiom of Mbr^^ inil trWii^ that iinp^ 
huMMn beings to mf^m, Ihit rcnseareh canwt mkA^ ^ cVarh how people 
stwmU feel in any <w instance. 
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lixrtfd as a random sample, all teachers of the ftwrtli grade in Farmer 
City, all parents of Nfiss Miller s sexentli grade? 

The merits of Inking explicit can Ih^ ilhistraled widi tMO correspondhig 
statements of a research prohlem. The <|uestion coiild be stated, "Might 
the first grade teachers of jasiXT Scliool, immediately following tlie first 
few weeks of ac<jnaintance with the children in thrir rooms, be able jndjR- 
mentally to list their children in order of readiness to read as acciiratelt 
as can 1k^ done throngli nse of the Metropolitan Readifiess Test?" Or- 
tainlw this <[nerv awakens more definite menial pictures tlian tlir com*- 
s|)onding qnestion, "Are teachers able to predict with jwme aocnracy the 
lK)tential achievements of stndents?" Tl\e first of thw two questions is 
obvionsly more limited. But it is concrete mmd specific it impkrs tfcr kind 
of test needed; and it is apparently doaMe. The secotid qur^on avokk 
mentioning which teachers, which students, what achiex t Tnenti, and mdei 
what ct)nditions. Its lxitmdarit*s are vaj^ie, and its empirical prferewts are 
lacking. 

M«Ofturobility. A third qtialificatiaBi for a researcliaWc qiie^n is 
that it shonld clearly define what is to be nneasured and Ao m M be icii^ 
to adeqnate methods of measurement. Tbis is simply to say that Ifcr «r- 
searcher who doi»s not know what t(vhnii|iies are availahfe to test Ifcr im- 
plications of his cjnestion is in a ixx)r [xwition to fonmilate nsabkr ifues- 
tions. The preceding cjuestion on readim^ss, for emmiple, identifies two 
ways to measure readiness— teacher judgnietil and the Metropolitan Bcwii- 
ness Test. It might 1h» that the accnracA iA these two ways of niiMfliiig 
readiness could \yv determined thnmi^ c^mifKirinfi the results thn fi\r 
with later scliool marks or achiev<*w»r«l Irtl ^cxwrs. Hnw wrB does %«r 
end achievement correlate with Kr0BniiMg-of-thc*-y<*ar |J*i4ktinii ^ 
achievement as obtained fiom Uitli te^rhrr jiidgnK'nts and ^iirtnipHhte 
test scx)res? Tliese addiiinnat measnrenHMit terlmiqiies suggest tin — In i> 
almost six>ntaneoudft'. 

The researchahle qnestion gi\ es clear aiKl nnawftikable clues to iprci- 
fic meanings: who. what. how. whct^ where, etc. Am omnihm queiliM is 
likely to Ixrget a puzzled answer. And a cpu qlMn tliat bdv waemm far 
securing an answer is certain to Ixjget fnistntio^ 

KVEIOPMKNT OP HYKIMMB 



In research, problems and questions som become transfHanA IhId 
hyix)theses. A hypothesis is a statewent or p a yuj ttion which is mmammd 
to formulate solutions to a problem. Whethrr the tentative pra^MMnn it 
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tlir S4>liiti«)«i to ten (lilHcnlty wliiili ori^inaUil tlic probU in is the task of 
tlu' nv|iiir\ . 

Tlir n>K' of tin lupotlu sis in siii iitific rc M uri li is to snj»jrrst cxpiaiia- 
tioiis tor UTtaiii fai ts ami to gniilo invc stigatitnis that will v^ rifv or 
rrfiitf' tlu' assnimil explanations and Kail to tlu' iliscovi rv of otliors. If 
till' «w^iginal prohlcn* is carrfnlly iK fiiu'iK tlic ilcfinition itsrlf will sugge st 
tiK' iirul of solution that is nniK^il. Whrn inv<'stigations of a prohh in arc 
CYHMMK-trd pnrrly at random, a mnltitmli' of facts will lu^ tnrmd up. hut 
thr^ m ill W so unrrlati d that tlu-ir very numluT will a<ld to the dillii nltv 
oJ rt*' prot)lem. It is quite j^ossihlc for a ri M^an h proj<rt to hv s\vamp<'d 
In lln' nwTv nmltiplicitx and diversity of facts, Tlu- search for cvid^Mic^' 
is hrsi caiHtiicted when the mo\t prohubk* meaning or supposition is us<m1 
as a ir„ii|<. ,n exploring facts-c spccially facts that would lead to on<' con- 
clitwn aiifcl rxclude others. 

To de»h{>e the genesis of a hyi^otln .^is. w<' will illustrate with a para- 
plir^'d vnmon of John Dewey's <v\aniplc of hypothesis making which ap- 
\H-mvi\ in km Uotc Wc Think. pul)lished l)y 1). C. I h ath in 1913, Consid<-r 
the familiar case of the automohilc that ix^rvi rsely dm-s not oi>erate prop- 
erly. Scmutliing is wrong, to Ik- sure. But how to i-orrect it cannot Ik- 
decided until xthat is wrong U-conu^s apparent. The untrained person is 
likely t«) make a wild guess or supiKisition and pn>ce<-d diri^tlv in random 
fashion to pake hcr<- and hannm-r there, hoping to hit ujwn the right thing. 
The trained person will pnKei'd in a different wa\ . He will observe with 
great car<\ ruling out certain iK>ssil)ilities and narrowing the range of what 
might Yw t\w Afficulty. He will use his knowledge of the structure of the 
maclmr tn fownulate the prohh ni and detect the trouble. He wit! proceed 
in a dtstwtly nonrandom mann< r. Finally, he wit! select his best guess to 
act upon, hilt he will not rule out further j^ossibilities by assuming that 
his liest giK*ss is certainly true. He n gards his guess as a guide, as a work- 
ing hyiK>tl»esis, and uses it to check some facts, estimate others, and 
gather new ones. He may think that if the trouble is caused by insufficient 
fuel, and all other recpiirements for pro\ iding fuel to the motor are satisfied, 
then most likely the fuel pump is pnn iding inadequate pressure, and he 
will look to sc^ if fust these conditions are present. His sense of the prob- 
lem lKXX>mes more adequate; the supposition ceases to be a mere hunch and 
becomes instead a reasoned possibility. 

The hypothesis implicitly states the problem and indicates the kind of 
sohrtion that is needed. It forms an inteHectnal bridge between the prob- 
lem and the tests of the most likely solutions to the problem. The hypothesis 
conditions the design of the tests, the collection of data, and the generaliza- 
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tions that may be drawn. Indeed, it is the pivotal point on whicli tlie 
whole research project tum^. 

RMMrch hypoHiMM ond null hypoHiMM. Hypotheses are fre- 
quently worded in "if-then" phraseology. A teacher may say, "If students 
choose the desks at which they wish to sit in the classroom, then they will 
have increased feelings of being accepted by the group." This teacher may 
derive his hypothesis from theories of social psychology or from the findings 
of other studies, such as J. L. Moreno's research into sociometric methods 
of grouping and regrouping. In such a case, the tueory or the study may 
suggest the specific kind of data needed (sociometric choices); the par- 
ticular information to be sought (increase or decrease in the number of 
mutual choices or isolates); and the design of the test (pre-test of socio- 
metric choices, then honest follow-through of regrouping, and a post-test 
at some later time). But "if-then" phraseology is not the only way to state 
hypotheses. Any other assertion is acceptable as long as it contains a pre- 
diction that if certain conditions are provided, certain results can be asso- 
ciated with the conditions. 

The kind of hypothesis we have been discussing is properly knovm as 
a research hypothesis. But if the study is statistical in nature, an additional 
kind of hypothesis is needed. This is known as the null hypothesis or the 
statistical hypothesis. The research hypothesis, as we have presented it, 
cannot be analyzed by statistical methods; the null hypothesis can be. 

The null hypothesis is sometimes known as a hypothesis of no differ- 
ences. For example, consider the question stated earlier about whether 
the first grade teachers of Jasper School can predict reading readiness as 
accurately as can be done through use of the Metropolitan Readiness Test. 
A statistical study might be planned to test the comparative accuracy of 
the Metropolitan Readiness Test and teacher judgment. The research 
hypothesis could predict: "Scores earned on the Metropolitan Readiness 
Test will predict readiness to read inore accurately than can the first grade 
teachers of Jasper School through listing their pupils from most to least 
ready following the first month of school in the autumn." The corresponding 
null hypothesis would be: There will be no difference between rankings 
of pupils in order of readiness to read as determined by the Metropolitan 
Readiness Test and by teacher listing.*" The null hypothesis is usually formu- 
lated for the express purpose of being rejected. If it is rejected, the research 
hypothesis may be accepted. 

The symbol used for the research hypothesis is Hi. The symbol used 
for the null hypothesis is Ho. When we want to make a decision about 
differences, we test Ho against Hi. In this case, if we let X stand for the 
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Metropolitan Readiness Test rankings and let Y stand for teacher judg- 
mental rankings, we can express the two hypotheses in operational form; 
Ho would he X~Y; Hi would be X > Y (X is greater than Y). 

The nature of the research hypothesis determines how Hi should be 
stated. If the research hypothesis simply states that the two measurements 
will differ, then Hi is that X # Y (X is not equal to Y). But if Hi predicts 
the direction of the difference, i.e., that one specified group will have larger 
or more accurate scores than the other, then Hi may be either that X > Y 
(X is greater than Y), or that X < Y (X is less than Y). 

In testing the assertion that the Metropolitan Readiness Test will pre- 
dict readiness to read more accurately than will teacher judgments (X > Y) 
—as measured by year-end achievement tests, for example— we cannot accept 
Hi unless we are able to reject Ho. That is, unless we can reject the 
hypothesis that predicts ''no difference," we cannot accept the hypothesis 
that predicts a statistical advantage for the Metropolitan Readiness Test 
scores. 7? we fail to reject Ho, then we must report our findings as those 
of no ait :ence. 

The decision to reject or not to reject a hypothesis is a function of 
statistical probability. The mathematics and uses of probability will be 
considered later in some detail. For the present, we can be content with 
this much consideration of problems and hypotheses. 



■■■■ The rcaMfchnr hccomet involved with poptilation* and Mmplet 
^^^H when he teU out to test a itatittical h\potliesi>. A itatiitiral 
^^■W question alwayi rrhtet to a group of individiiali, ratlier than to 
1 • Individual. It aiki .what it tnir of tlir group. Stali»tical 

inquirief are of two types. One type of im|Mir\ calls onlv for 
a dcKription of the group of indi\iduals i^uall> obsened. Suminarv 
meaturet, per cents. a\erages, or expressions of \arial>ilit\ are computed 
from the obaervatiom. These measures comprise wliat is known as df- 
•criptive sUtMici. 

A second type of statistical inquiry is more concerned with scientific 
investigation and utilizes inferential statistics. It in\ol\es a search for ideas 
and principles which have some degree of generality. If an investigation 
deals with matters of braad interest, the findings are usually applied to a 
much larger domain than the cases actually observed. This larger domain 
is called the p o p u kh on. and the group of cases observed is called the 
mmpk. The statistics obtained from the sample, which express crrtain 
characteristics of that fpoup, are used to obtain information concerning 
the unbwwn group chan^teristics of the popiibtion. Such generalization 
from sample to population is known as ttmhmical Inference. 

**'' * *t* pa l inf«rc«oe is concerned with two types of problems: estima- 
tioa of popttktion parameters and tests of hypotheses. While our primary 
concOTn in this book is wHh tests of hypotheses, we stiU need a clear 
uwfentanding of popubtion and sample. 

In everyday language, the term popttl^km is used to refer to groups 
of people. We sneak of the popubtion of Chicago, or of the sUte of Uuh, 
or of ttw United States, meanhig by this all of the people who occupy a 
deinito gaographical space at a particular time. The statistician, howe\ .>r, 
empbys this term in a more general sense to refer not only to defined 
groups of peopb but abo to objects, materials, and measurements and to 
"thtupr or " hap penl n g i '' of any kind 

For eiampb. we may be interested in children with IQ's bebw 90 
who bebng to a high socio-economic group, or we may be interested in 
children with IQ's above 120 who bekmg to a km socio-economic gro«tp. 
If questiom are raiwd about the proportion of children in a particubr 
popubtion or subpopubtion with IQ's above or bebw a specified value, 
or if questions are raised about the rebtionship between IQ and socio- 
economic level, then these are questions of a stotistical nature which deal 
«irith specified popubtkms. 

A further distinct i on may he made between infiiUte and finUe 
popubtkms. The limttless number of times a die oouM lie rolfed. 
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or all of the variations that could be found among Arctic mas(|uitoes are 
examples of infinite or indefinitely large populations. And there are manv 
populations which are actually definite or finite hut which are so large that 
for all practical purposes they can best be regarded as infinite. The ISO 
million or so people living in tlie United States constitute a large hut finite 
population which for most statistical purposes may be assumed to be 
infinite. This would not be true of a deck of cards, which may l>e thought 
of as a small finite population of 52 meniUTs. Nor would it Ih* tnie of 
the twenty-three childrer. in Miss Butlers fifth grade class, if they were 
defined as a finite population for a particular purpose. 

If wp had a research question concerning \liss Butler's fifth grade, 
wc probably would prefer to study tlie wliole small (finite) [xipulation, 
and we would not have to draw a smaller sample from it. But of course, 
we would be able to make statements only about this particular class and 
not about fifth grade cbsses in general. 

Miss Butler*s fifth grade, however, migiit be considered as a sample of 
all fifth grades in \wr particular scIkm>1 system. The extent to which we wi?:li 
to generalize our findings from sample to population plays a major role in 
tlie procedures used. It may be assumed that any group of fifth graders 
is more like other groups of fifth graders than, let s say, like set*oud graders 
or twelfth graders. Miss Butler might consider this similarity among fifth 
grade groups as a rough approximation of a sample drawn from a larger 
population. Subsequently, tlie accuracy of this assumption can Ik> chal- 
lenged against test data averages drawn from many fifth grade groups in 
the same scIkwI system. Miss Butler can use statistical tests of distribution 
very handily and profit from the knowledge which can l>e obtained. In 
addition* she can generalize lier knowledge considerahI> bi>youd her par- 
ticular class group. She can ask a question to determine whether and to 
what extent slie would be justified in considering any |)artieular fifth grade 
class auigned to her as a sample of all fifth graders she has taught in the 
past or might teach in the future. Slie could quite easily test the hypothesis 
that succeeding fifth grade groups in the school wliere she teaches will Im* 
significantly similar in crucial educational characteristics by appl\ ing the 
chi square test of the significance of differences. Of course, this would 
result in a kmgitudinal study stretching over as many years as slie wanted 
to go, but it could be done provided both the foresight and patience 
were there. 

But if for some good reason it became necessary to arrive at general 
statements concerning all fifth graders in the State of New York, then we 
would have a much larger problem. Altliough the population would t>e 
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finite, for the sake of economy and reason, it should be treated as infinite. 
We most likely would want to seleci a representatixe sample of fifth 
graders or fifth grades, measure tlu'm in relatioii to crucial concerns, and 
infer back to the total population. 

SnJCTINO A RVtiSINTMIVI SAMPU 

There are three major steps in sc'lectinu a representati\c sample: 1) 
determining tlie nature of tlie iMipnIation; 2) selecting a statistical model 
for analysis of the data; and 3) clioosing a nu'thod for sc^h^cting i\w 
sample to make it as representative as possible. We will now briefly 
consider each of tliese steps. 

Pfrmlwifif Hm notaira •f fli^ p^puhrtiM. In order to determine 
the nature or shape of a population, we generally rely on a body of 
knowknlge from matl)ematfcs. We know that wlien eiKMigh measurements 
are made of any natural plR*nonKMion— tin* size* of oranges, weight of cats, 
height of adnlt human IxMugs, IQ sct>rcHi»~and are charted on a graph, tliey 
tend to form a bell- or sombrero-sha|K'd curxe. This ciirxe (or dislrilxition ) 
has clonic to l)e known as tlie normal curve of proMMUj or, more simpl\ , 
as tlie normal curve. 

Tlie normal curve charts the dispersion and variation which can be 
expected in any "nonselected" |x>pnlation of mc*asnreuMMits, wliere chance 
alone is operating. It represi*iits what will In* the distrilMition of measure- 
ments or observations chie to the* prokibilit) of <x*cnrrences. It pictures the 
theoretical result of all chance distributions. In scImh>I |)arlanc*e, iW* normal 
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curve is used to express tlie distribiitioM of IQ scores standardised achieve- 
ment test scores, lieights, weiglits, and niinH*roiis otlH*r familiar nueasiire- 
ments. And it is worth noting that tlie mr\e teiuls to snKxHh out and to 
become more bell-shaped and syminetricai as tin* nimilK*r of cases is 
increased. 

Because of the steady reciirre»ct» ol the normal curve when brge 
numbers of plienomena are measured -^tid cliarted, we are tlieoretically 
iuitified in assuming tluit large or infcw u^ i po|>u)ations will also take the 
same distribution. Sometimes, in re«e«uc i, studies, we are able to test this 
sort of assumption; more often, it is simpiy anmmed to be true, without 
test. This practice of operating on tlie kaiis of reasoned, and not tested, 
assumptions is characteristic ol statistmii procedure, even when popula- 
tions are not indefinitely large. 

S ilirttog m il Hl il k «l muM *>r w I h Ii #1 fkm Wearefaced 
with many of the same pcoMrmf when we seek a statistical model or test 
for analyzing the daU obUined from a sample. Customary parametric 
statistical testa of pfobability, such as the t or F distributkms, are based 
on their own strong assumptions concerning the shape of the distributkm 
which is assumed to characteriase both the parent popuktion and the 
sample. Prominent among these assiimptkms is that the observations are 
drawn from normall)' distributed populations: they must fit the normal, 
"chance" curve. The more numerous and stringent tlie assumptkms, the 
more questionable the decisions arrived at throu|^ the use of any statistical 
test of probability. We know that many populatiom or Minplei are not 
distributed normally. What ihen? What happrm when the measurement 
is not of sufllcient strength, not truly numerical? What sort of statistical 
model might we select which is not hedged with strong assumptions about 
the shape of the distribution and measurement requirements? 

There are satisfactory answers to these que^ions. just as the para- 
metric tests are based on distribution and measurement assumptions, so 
there is a branch of statistics known as distributkm-free, or non p wu m afrfc, 
statistics. With these nonparametric models we may say, ''Regardless 
of the shape of the distributkm, we may observe that the probability of 
occurrence is ... . 

However, we should note that the paramefric tesU with the s t ro nge st 
assumptkms are also the most powerful tests. When those assumptkms are 
valkl, these tests will be most likely of all to reject H. when H. is false. 
But the assumptkms and requirements underlying these mote powerful 
tests must he met before confidence can be pliced in any probability 
statement. 
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We will sliortly return t<» flu* use* of distrthiition-fn'e statistkal in<Hl(*ls 
when we ctinsider the aiiak«is of data oMained fnnn |M)|w4(itioiis and 
samples. 

is a iK*cessarv prcxediire i ji^mtm of wliiOii r nv ;in* n)iiC(*i »M (l witli the 
shape* of the parent |K>pulHkM Tlie saiiir |f««iH*€*ilur(*s ol saii*|)lin^ appK 
to all cases wktrre the ffrmsmtmibtnr in tlu» pi^Hfcation ean \:ilnll\ lu* rediietMl 
to promote i^ne and com i—— 

Tliere an* manv wuvs v» linam a sniaAn- Kaniple irtmt a larger i^roiip. 
Biit in all tlie many diffrMr tvpes ol saiHfries. tin* iiw»st inii^N-tant eliai 
acteristic is rmndomnens, ^ t^m \s meant tkat eviTv siin^k* sani|>lin^ nnrt 
in the population has an ecpaA c^kh* of \wwi^ clrawn into {\\v sam|>lt<. In 
tlie mmt direct form, the mm^i^ mmgSxX U* drawn hy siniph writinu on a 
cardtiN* name of each indtviAMl Mirtiided in ^ fnilo jiopidation, M ramldinu 
tlie canli in a l)Ox, and tWn i^^^-^ting, williMit looking, tin* nnniUT of 
names desired for the sam^ 

Snmetliing like this wa*i4t m <M*k*i*tiiii|i; men to Ih* ealKni f(»r military 
ser%iee during World War II tin* NatMmal Seleeti\<' S<»r\iee Ait. 

When each individual I «^ his Ibt al draft lioard. lu* was a5»sipM*d 

a mmiber, ranging frw mtt to w^rral •iMnNnwl Kadi of tlieM* nnmlM*rs 
was tlien encased in a cafmi^k wbdr gmNp of ea|>sules tlionMi^d) 
mixed, and all vi tliem ifkmtm m a largr fkliktml. TIm* Prc^sident of tin* 
United States, hiindfoldnl, ruarlied into IIm* Imwl aiui drew out one capsule 
after another. TIh* local draH hoards sc»nt out imluetion iMiticc*s in tlu* 
order of the drawing to tin* men Inilding tin* S4*lectcHl iniinliers. Placing all 
tlie numbers in similar capsules, |i«itting tliem in tin* saim* liowl, mixing 
them, and tln^n hlimlfolding tlie drawer, madi' this ty|M* of sampling truly 
random and nonhiased. SikIi a |)rocedure is kinnvn as nhn^^e random 
mmpt^ng. In using this procedure*, ever\' effort is made to reduce the 
possibility tliat selection of tin* samph* will he influenced hy either 
conscious or suhc<Hisc*ious l>ias and to lea%'e instead tlw* decision solidly to 
slieer chance. 

Questions about tlie optimum si/e of tlie sampk* in resc*arch logically 
arise at this point. Statistical calailations can give us an idea of tin* degree 
of confidence whicl. mn be plac(*d in tin* accuracy of a sample of a given 
size and of the sampling err.ir which may lie present. By and large, liow* 
e\'er, the accuracy of iinv sa^npk* can only be estimati*d. Tlie rough rule of 
thumb is that small pa y iAitimis revpiire a pe u f i w l ionally larger sample than 
do brge popidations ¥m f&ampir, it Mght be preferable to draw a 90 
per cent oi larger ^ the HM) sMk graders in a partictilar school, 
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while a 10 pvr wnt or sniallt r sample of the 74,500 sixtli graders in a par- 
ticular state might Ix' adequate. In Ixitli cases, random selection would be 
necessary. 

Sampling from popuhitions of very large size can become inordinately 
time ccmsuming and tedious. Easier ;nid defensible results can be ob- 
tained by use of a table of random niinilKTs such as may be found in 
most textbooks on statistics. TIm^* tables wntain sets of nnml>ers that 
after careful examination ha\e slwmn no evidence of systematic order. 
Before using tiw table, it is nt^v^v^ to nunilKT all the individuals in the 
population to Jh* studied. Tlwa Im> done starting with 1 and proceeding 
to 50, or 99, or 199, or 1999. or vikiitever figure equals the total population 
number. The table is then ent(>rrd at some random starting |K)int-perhaps 
by making a blind stab at tin' page with a |K>ncil-and the cases whose 
numbers come up as one moves down the colunui of numbers are taken 
into the sample until tlie desircnl numl)er of cases is obtained. The selection 
of any given case platvs no limits on what other cases can be selected 
from the population. 

Otlier ty|>es of sampling are available, such as Mtraiified random 
sampling. This approach uses the same "blind but fair" techniques which 
cliaracterize random Simipling. But when a population is composed of 
several subpopniations-for example, Negro-white, cnlturallv advanced- 
handicapped, or intellectually ac^elerated-retarded-it may be divided into 
two or more strata. A random sample is taken from each stratum, with the 
subsamplc^s joined to form the total sample. There is no attempt to make 
stratified samples a replica of cmres|M)nding |x>pulations; the intent in 
stratified random sampling is only to take into acc^ount the anticipated 
liomogeneity of tlie defined strata with res|xx*t to the characteristic which 
it being studied. And of course, each of the subsamples can be investi- 
gated separately. 

SMCIM PtOMJMS IN SCHOOLS 

Knowledge of |M)pulation and sampling techniques is valuable to the 
extent that it can \w used feasibly and accurately. But the conditions 
•urroun/l«ng scImx)I». their organization, and the teaching that goes on 
witliifi tiic*m set rather severe limits on choices of populations, to study and 
on the kind of samples which can Ix' drawn from them. Some attention to 
these imposed limits and restrictc*d choices will ramplete our brief treat- 
ment of popidations and samples. 

In tlie first place, the students in any single scliool or scliool system, 
and tlie teacliers and administrators tiK rein, are not examples of natural 
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or unsexcted population. They represrnt hif^ily selected and particular 
kinds of captive pniyi Furthrrmore, the kmprr children are in school, 
tkr keener becomes the quest for siirvf\al mmi the larger becomes the 
aMmuatian rate, thus leducing the number of «tain types of children in 
mgular aiamdanoe. Jmt witness the national t— rrn over sdwd dropout 
nies. Mnre than tim, iht various ways in wAmh children are fpm^ped, once 
ikry enter schools, mmah in quite unnatanL nnrandam ampegates of 
indiividuals. The Mb variations in fmnping prttomn employed 

among nmnbers ml i^nrate schools and the select i^nanam on theit 
populatiom greaiK omiplicate the process oi generaliamK tmm sample 
to univene. 

A further onmpkcaiiMi arises from the fact that mimnb me deliben^, 
goal-oriented soaial inaHMions, remarkably insulated iami aoimrtttc manip- 
ulation. To a fl§nifionnt degree, researchers in ednmimn am limited in 
the extent to which thay can experiment with fke o^imiartim of grade 
groups, with imfeuctiaaal requirements and the allocniiBB of mntent, and 
with other pohcms ahaut the conduct of the school, la other words, 
researchers dealing witii the schools— with delibemte, gaal-anonled institu- 
tions— must to an important extent woric with the gronps that ihey alnady 
have. Because of the very natare of the sdmrf as an inalllntioii, they 
cannot select different group* o^t of a grots, chance population and begin 
afresh to organize an imaginative educational system u n n poaed of chance 
students. 

Coincident with this observation, it is impamible to avoid thinking of 
the Amidon School in Washington, D.C. Here was a once-in-a-lifetime 
opportunity to choose the students for a school without a contiguous group 
of parents, house them in a new building, and manipulate the instructional 
program. Of course, this opportunity came about through certain miscaku- 
btions in urban redev «.kipinent which remked in a new school buikling 
without a comnmnity and without children to be educaiad. There was no 
other chok» but more or less to manufacture a student population by 
importing children from other areas of the city. What a wonderful chance 
this would have been to undertake some careful, imafhuitive, scientific 
researchi 

In any event, the particular problems of population and sampling in 
rehition to the schools may be responsible for the tendency of educational 
research to deal with the '^samples*' (class groups) which are already there. 
Such "samples** are assumed to be represenUtive of a burger population of 
theoretical class groups, and findings are then generalized from samples to 
populations. And then teachers and administrators are sometimes surprised 
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PThe central oiitlino of an oxpenmont is simple. An experimental 
group is ex|)oseil to the ('X|>eriinentai \arial)le wliik- a control 
group is not; the two groups are then (*oniparc*ii in temis of the 
ol)ser\eil effect. It shonlil Ih' stressed at this i>oint that mmiic 
sort of a control group is essential in ex|>c*rinM'ntal research. 
However, practical limitations freipieiitly liemand nioiliHiations ami in- 
%'entions in establishing experimental controls. 

We have jnst discussc^il some of the diffienlties invoKeil in Minring 
adeqnate jxipulations and samples. The ;ittenipt to provide rcjEMin.imt* 
similarity Ix^tween exiKrimental and in)ntrol gnN»|^ iiinnediateK doiit4r^ 
the diffienlties. Ft)r the pnqxises and conditions of most MK'ial rcM-^m^h. 
the "elegant" research design with "pun* e\|KTiimMital uikI control gvmpai. 
nicely matched in all im|)ortant respects, is selcknn fc^Mtlikv Rather tiiM 
lose the i^otential knowledge to Ih» gained tliroiigh exp^iiMental reM»c*h 
we tend to deal with estimates or appniximatioiis almit tin* nieanine <«t 
onr data. It is one thing to rc*cogni/e wiuit the ideal sH otf conditions sm 
ronnding research may Ik*; nu^eting tkc-se conditions m tlu^ ordinary, on- 
going practical world is something else and fre(|neiitK calls for a gcKxI 
deal of imagination and inventiveness if tlie ideal is to \ie even approached. 
An nnderstanding of variables and proluibilitx' is im|)ortant e<|nipment for 
dealing with the realities that often complicflie l wpiiini id researdi. 

INOmNDiNT AND DMMWT ¥MMUS 

In research, there are two broad kinck of variables to be reckoned with. 
First, there is the experimental variable which is the crncial characteristic, 
occnrrence, or procednre Inking inx estigated. This is also called the tWe- 
pendent variable. Cnstomarily, the experimental or independent variable 
is symbolized by the letter X. Second, tliere is the drpcnJmf-or criterion 
or prer/<rfer/— variable which is the assumed effect ef tlie independent 
variable. It is called the dependent variable becaiwe the prediction of it 
depends on the functional relationship or value of tlie mdependent variable. 
Cnstomarily, the dependent or criterion variable is symbolized bv the 
letter Y. 

Thus, an experimental hypothesis asserts that a particidar character- 
istic, occurrence, or effect (X) is one of the factors that can be associated 
with another characteristic or occurrence (Y). Or, the hypothesis mav 
predict that if X, then V. Studies designed to test such hypotheses must 
yield data which will lead to tlie inference that X c*an or cannot be 
associated with Y under specified conditions. 
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An experimental design provides Iwth greater assurance and greater 
efficiency hy making jwssihle the simnltaneons collection of several lines 
of evidence. More specifically, the use of ex|X'riniental designs makes 
possible tlu' collection of three major types of evidence necessan* to testing 
hypotlieses; 1 ) evidence of concomitant variation—that is, evidence that 
the independent variable (X) and the dependent variable (V) are asso- 
ciated; 2) evidence that the dependent variable (V) did not (K'cnr In^fore 
the independent variable (X); and 3) e\ideuce ruling out other factors 
as possible influences on the dependent variable. 

EvUmm of cofKpmitont vorioNon. Evidence of the first type is 
jjecured very easily in an experiment. The investigator knows which sub- 
jects have been exposed to the ex|>erimental variable (X) and which sub- 
jects have not. He measures all subjects in terms of the dependent variable 
(Y). He can then determine whether Y occurs more frec|uently among 
those Mhfects who have been exposed to X than among thosc> who have 
not, or whatever speciPc relationship is predicted by his hypothesis. 

KMmim thot th« d^pudnf variobto did not occur boforo tho 
ifido^OMloilt vorioblo. The second type of evidence needed to test a 
hypothesis is secured in one or l)otli of two ways. The investigator can set 
up the experimental and control groups in such a way that it is reasonable 
to assume that they did not differ in regard to the dependent variable 
before the experimental group was exposed to the independent variable. Or 
the investigator can measure both groups in relation to the dependent 
variable before one group is exposed to the independent variable. This 
results in a "before" measurement and an **after" measurement, or in a 
pre-test and post-test type of design. The differences between the pre- and 
the post-tests for both groups provide the desired c*omparison. The intent 
behind this requirement is not to demonstrate that one or both groups have 
none of the prior knowledge, or attitudes, or qualities being investigated, 
but to demonstrate that the experimental group did not have more of the 
factors being experimented with before the start of the experiment. 

Evidonco ruNng out othor foctort. There are several ways to obtain 
evidence ruling out other factors as possible influences on the dependent 
variable. At least four major sources of uncontrolled factors may interfere 
with an experiment. 

First, the measurement process used in the experiment may affect the 
outcome. If people feel that they are being used as guinea pigs or if they 
feel that they are being tested and must make a good impression, or if they 
feel that they are different from other people because they are defined as 
experimental, the measurinic process itself mav distort the results. The 
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"before" measuring may mstallize attitudes related to the experimental 
variable. The "after" measuring may Ix^ influenced if the subjects reflect 
test fatigue or become bored or defensi\e. The subjects ma\' try to keep 
their second responses consistent with their first responses or mav try to 
vary them if they feel that variation is expected. If there is reason to sup- 
pose that any of these influences is present, alternative designs can he 
employed to lessen the impact of the measuring process itself. 

Second, normal changes related to the maturation and development 
of the subjects may Ik* c*oiifiised with the results of ex[)eriniental treatment. 
This kind of unplanned interference is es|)ecially critical whenever an 
experiment extends over a long period of time. However, if the maturational 
changes can be assumed to be approximately the same in the experimental 
and control groups, the effect of maturation can be ruled out bv comparing 
the two groups. Sometimes, interim measurements during the course of a 
long-range experiment will indicate whether the two groups are becoming 
noticeably unlike. 

Third, in any social ex|>eriment, unplanned contemporaneous events 
may affect the experimental outcome. If during tlie course of an experiment 
in the teaching of science, a foreign nation were to claim new and frighten- 
ing scientific discoveries, attitudes toward the teaching and learning of 
science in American schools might be influenced— as indeed thev were in 
the familiar instance of Sputnik I. If such an event were to affect the 
experimental and the control groups in the same way, no problem would 
be created since an effect common to the two groups could not be a cause 
of difference between them. However, it is very difficult to ascertain the 
relative impact of deep-seated, unplanned events on two separate groups. 
Such events cannot be controlled, either In tlie sense of holding them con- 
stant or in the sense of deliberately manipulating them. But some experi- 
mental designs have been devised to take account of this possibility, at 
least in part. 

Fourth, characteristics of the suhfects that are relatively enduring or 
are the result of past experience may introduce systematic differences be- 
tween the experimental and control groups. In order to nde out such 
differences in relevant factors, the investigator may set up his experimental 
and control groups so that it is reasonable to assume that they do not differ 
widely in the distributions of age, sex, intelligence, social background, 
academic achievement, and other factors seen as relevant to the experi- 
ment. Or he may measure them before the experiment in terms of such 
factors; or he may do both. All of these factors relating to subjects are also 
"variables." But they are controlled variables, controlled to eliminate their 
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influence on the dependent variable which is Ix^ing studied. At times, 
groups ratiier than individuals are matched. For instance^ it may )>e suffi- 
cient to make sure that an experimental and a control group are similar 
in relation to averages and ranges of age, intelligence, achievement « and 
social background, and any other factor presumed to intrcKluce systematic 
differences. This practice is known as frequenctj distrxlmtxon control. 

STATISTICAL TESTS OF PIOBAULITY 

It may be that we have carefully allowed for the collection of these 
three types of evidenc^e necessary to testing hyi>othese.s, hut there still 
remains one more type of evidence which must Ik* ohtaiued. Without it, 
we still might Ijelieve that we had established a (*l(*ar relationship In-tween 
X and y when, in fact, we had not. It might easily 1h» that V (Kcurs as we 
observe because it is an accidental or fortuitous (xvnrrence. This fourth 
type of evidence is scx^ured through statistical tests of probability. 

Probability enables us to allow for the intrusion of sheer chance or 
sampling error. The extent to which chancr might have entered into a 
research study is expressed as significance. If chance might ha\'e playi*d a 
large part in the findings obtained, the significance value. express(*d as a 
percentage, will be large; if chance played a small part, the significance 
value will be small. 

Tiie problem of deciding on the significance of research findings re- 
lated to schools may be directly associated with the fact that almost any 
teaching method or organizational practice is lK>und to result in some sort 
of ciiange in the students ex|)osed to it. Most teachers and administrators 
know that it would be extremely difficult to prevent students from changing 
(learning) to some extent, regardless of the approach used. This is all part 
of the dynamic nature of human Ix'ings; they simply will not stay still. 
Even if a student is left alone, he will ctmstruct an activity out of some- 
thing, if for no other reason than to have something to do. Because of this 
incessant momentum, there is always the real possibility that the gains in 
learning observed by the teacher may have resulted from sh(*er cliance 
happenings rather than from the activities deliberately planned or assigned. 
And to complicate matters still more, the students of any class, in any learn- 
ing activity, will show differences in (juality, (piantity, and speed of learn- 
ing. Some may give considerable evidence of gains in the desired direction, 
and some may even show a loss. How then can we determine whether the 
results of any research project related to schools, children, and teachers can 
Ix* attributed to chance or to the effec*t we predict? 
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AN NXUtTRATION 

Lrt'i %ve how prohabiHiy and tests of significance are used in experi- 
mental researcli. Take the case of Mr Black, sixth grade teacher, who 
wanted to experiment witli wa\s to increase the number of Hhrary hooks 
read by the students in liis room He felt that the students would read more* 
lKx>ks if they had the additional motivation and rcx^ognition that could 
come aU)ut through writing and puhlishing Ixxik reviews with a In-line 
in the class nc*wspapf>r 

Mr. Black began by consulting his records to determine the numlx r 
of liooks read \ivr month by each student. Then he listed the students* 
names in order, from most to least books read per month. After planning 
with tlie class, lu encouraged them to write reviews of the lKM)ks they read 
and proceeded to c*ompile anotlier set of records. Two months later, he 
again listed the nanu^s in the sanK' way lie had done U^fore to determine 
whetlter the prc^dicted iiKTeuse in reading had occurred. 

In folkiwing tlnse procxnlures. Mr. Black had used a pre test and 
post tes*. ^luulr group design which utili/cnl sixth grade students as their 
own controls. That is, the pretest (number of books regularlx read) 
served as t\w control group, and t\w [lost-test (number of lxx)ks read foh 
lowing tin* writing of leviews) served as tl»e exiXTimental group. He felt 
confident that he liati given ade<piate attention to concomitant variation, 
tliat lie had evidence of the fact that increases in reading (Y) had not 
occurred before ihe writing of reviews ( X), and that other factors had lieen 
controlled by presenting tlie proiect to the class in a calm manner as a 
part of regular, everyday clasiroom work. 

But he still was faced with tlw^ neceuity of accounting for the opera- 
tions of pure chance. Mr. Black observed that of tlie twenty t>ight students 
in tlie room, eighteen liad increased tlu^ir reading, three had read exactiv 
the same nunilier of Imoks per mmth that tliey liad read before, and seven 
had read fewer liooks. He couW see that tliere liad been a cliange for the 
better with t\w eighteen students. But what about the ten wlio showed 
eltiRT no cliange or a kw? Were tliese miiks significant, or coiiM this 
distribution liave btvn tlie result of slieer chance? Unless lie ooiild deter- 
mine wln-tlier tlie results were s'^nply cliance findings, \w cfwild not deckk> 
wliether lie liad made significant discoveries about ways to stimulate hi% 
class to rf*ad more librar\' books. 

TIkw are wry gotnl, feasible metliods for Mr. Black to use in deter- 
mining tiK' significaiKv of his fiiMhngs. He can tuni to some of tlie prolia* 
bilitx tests tii statistical infereiK-e, ap|ily his observations (data), and 



EKLC 



Re^^earvh for thv Practitioner in Educatior^ 



dotermine whrtluT i\w results arc statisticalhj significant. His aims are to 
judge the significance of the data and to make maxiniinn use of the infor- 
mation gathered. These are the principal proNems with which statistical 
methods are concerned. And Mr. Black, sixth grade teacher, can learn to 
use the statistical tests he needs in just a few weeks. 

Whot "probobility" nwons. It will Ik* helpful to take a moment or 
two to look more closely at what Mr. Black would Ik* doing if he applied 
an inferential test to his data. He would start with the realization that a 
statistical cpiestion always relates to a group of individuals rather than to 
smgle individuals and reveals information al>out the characteristics of the 
group. Next, he would know that the results he would derive from the 
test would take into account the probability of chance occurrences and 
enable him to say, with a specified degree of confidence, whether the 
differences he noted in the numlKT of lKX>ks read could legitimately |>e 
attributed to the writing and publishing of reviews in the class newspaiH*r. 

He would realize that the results of the statistical test would be ex- 
pressed in terms of the probability that bis decisions might )>t* wrong. 
Even if be finds evidence to indicate that writing rexiews is an effective 
stimulant to reading books, he must allow for probable error in deciding 
that this is so. There are \ery few facts in life alxnit which wc can bt' 
absolutely certain. We are far more justified in thinking in terms of 
probabilities than of certainties. 

For example, insurance actuaries have calculated the risk of people 
in the Unitcnl States dying at various ages. Their figures let us know that 
a twenty-year old person has one chance out of approximately 740 that 
he will not see his twenty-first year, and that a thirty-four year old person 
runs tlie risk of one chance in approximately 440 that be will not become 
thirty-five. However, if we were to make predictions that a certain num- 
\yer of any particular group of twenty-year olds or thirty-four year olds 
would die during those years, we would probably be wrong to a certain 
extent. Yet if our predictions were consistently more right than wnmg, 
we might regard them as statistically sigi.ificant. 

But of course, Mr. Black would not be dealing with insurance tables. 
He would be using the mathematical materials and tables which have 
been worked out to express the probability of occurrence, such as the 
chance of drawing one of the four acrs in a game of poker. (Paren- 
thetically, we might make the observation that all statistical procedures 
originally grew out of the study of games of chance.) But to continue 
with the explanation of probability: If two or more events are equally 
likely or probable of occurrence, this fact can be represented by a ratio 
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—for rxainpU*, 1 2 as in coin tossing, 1 6 as in throwing dice, or 1/52 
as in drawing any particular card from a standard deck. The ratio 
means that if there are .V possihihties, the prohahihty of the wcnrrence 
of anv one of these ),)ossihihti(\s is 1 \, Thus, if a coin is tossed, the 
prohahihty of its faHinj^ heads up is 1 2, and the very same vahie holds 
for the prohahihty of its falhng tails up. And since 1 2 '*' 2—1, tliese 
two possihihties are mathematically and logically exhaustive. Similarly, 
if a die, which has six sides and thus six possihie ways of landing, is 
thrown, the prohahihty of any one of the six sides coming up is 16, Six 
times this fraction ecpials om\ which again is logically exhaustive. 

From this way of stating prohahilities, all manner of more complex 
expectations or prohahilities can he derived. For instance, the expectation 
of ohtaining two heads from two tosses of a coin is no longer 1/2 but is 
1 2 X 1 '2 or 1 4; the expectation of ohtaining three heads from three 
tosses is I 2 X 1 2 X 1 2 or 1 8. Or the exiK'ctation of rolling two sixes, 
or anv other two numlx*rs, on a pair of dice is 1 6 X 1 6 or 1 36. And 
it has been found that if all possible combinations of heads and tails on 
ten successive tosses of a coin or on the simultaneous toss of 10 coins are 
computed, the resulting distribution turns out to be the normal, bell* 
shaped curve. 

Whot ''tignificonM" mMns. Frcmi the type of mathematical mode! 
just described, it is possible to say just how likely a given outcome is if 
chance alone is operating. If, therefore, in a situation where an inde- 
pendent variable lias been deliberately introduced the results are such 
that they might easily be attributed to chance alone, we do not consider 
that anything has been demonstrated. But if a result is obtained which 
would l>e expected by chance only one time in a thousand ( P = .001 ) 
then we can say that it would be very unlikely for chance, rather than 
the variable in question, to be influential. This is the logic of the null 
hypothesis— the hypothesis that the difference obtained between the ex- 
perimental and control groups is not due to the' deliberately introduced 
independent variable but is due instead to an accident or error of sampling. 
What the evidence we collect does is to discredit the null hypothesis— the 
hypothesis of no deliberately produced difference. It is here that the 
calculation of probability may help us to decide whether we have achieved 
evidence in which we can have reasonable confidence. 

It is just this sort of calculation and decision that Mr. Black would 
undertake in analyzing his data for significance. Fortunately, he has 
access to statistical formulas and tables which will simplify the computa- 
tion for the various tests he might use. And he can use the newer 
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distrihution-froe tests wliicli haw Ikhmi madr goiu-ralK availal)lr siiui' 
1956, In using any of tlir tests, Mr. Black would have some decisions of 
his own to make. He would have to decide how much risk he is willing 
to take that he might Ik wrong. Would he Ik- willing to accept his experi- 
mental results if he were wrong one time out of a hundred? Five times 
out of a hundred? Twenty times out of a hundred? Impending on the 
crucialness of his research question and the accuracy of his measuring 
instruments, how right might he insist on being, and liow much risk is he 
willing to take that he might Ix* wrong? 

In the language of statistics, probability values range l>etwe<Mi 1 and 0. 
The probability that each of us will somedax die is I. Tlir prohabilitv 
that any one of us will never die is 0. The probabilities that people will 
live to given ages may be expressed as a decimal fraction somewhere 
between 1 and 0. All probabilities may Ix' expresjK?d this way. For in- 
stance, if we toss a coin, we can state that the probability of obtaining a 
head is one out of two. We write this as F ^ 12, or .50. 

Mr. Black will state the risk he is wuling to take this wav. If be 
wishes to restrict the probability of being wrong to one chance out of a 
hundred, he will test his data at the ".01 level"; if he wishes to restrict it 
to live times out ol a hundred, he will test his data at the ".()5 level," and 
so forth. The statistical tables he will use are designed in terms of these 
alternative choices, and he can read directly from them the significance 
of his data at the level of risk he is willing tD take. Of course, if he chooses 
to work at the .01 level, he will not be able to accept his experimental 
findings and consequently be able to reject his null hypothesis as fre- 
quently as he might be able to do if he worked at the .05 level. And he 
could possibly reject the null hypotliesis many, many times more fre<]uently 
if he worked at the .10 level. But the decision is Mr. Blacks, and he 
should report his research findings as "significant at the % level." 

Then other people can decide whether, for their purposes, they wish to 
accept or reject his findings at the probability le\'el he reports. 

Working with these statistical methods, Mr. Black would know that 
significance is never a matter of true or false, of significant or not signifi- 
cant. He would know that significance is always expressed in levels of 
probabihty of occurrence in an infinite number of theoretical tries, stated 
as a percentage. 

Incidentally, the table of probabilities for the distribution-free Sign 
Test shows that for the way Mr. Blacks students were distributed in 
regard to increased reading of library books ( 18 read more, 3 read the 
same number, 7 read fewer books), the experiment was significant at the 
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.022 SluMT c liaiK'<* coulil not lia\r iUioimtnl loi tlirsr irsiilts more 

tliaii 2 * tiiiHs out of a liiin(ln*(l tlMM)irtjcal hirs. 

W'v slioiiM v\h\ (Mir lonsidirahnii nf prohahilitv willi a woril ol 
rivstraiiit. Willi a larj;i* <»ii(>iii;li iimihImt of cum s. r\tri'iiirl\ siiiall ililfrr- 
vui'vs nia\ Im* "slatisUtalK siynifkanl ' F</r rxainplr. witli a Mifficiriitl\ 
larj^r rxprriiiuMiUil ami toiilrol i;ioii|). a ilifli rriur of our point lu'twi rn 
tlirir iiuan or iiu*cliaii I(^ .scor«s tonlil iiiakr tlu' iliffi iriKc 'significant/* 
altlionj;h it is liar<l to svv what iniportancf sue li a small iliffi rnu r iiii^lil 
liav(*. It is sonirtinirs difficult to nrogni/r any iral import in trivial (iiul- 
iiigs. Statistical tiiaiiipiilatioii is no snhstitntc for tliinkiiii^. 



HljjlfH The various phases of the research process vau he hroiii^ht iiiuler 
^^^H^l control In designing the research. To desip) is to plan; that is« 
^^^^^1 design is the process of making decisions lH*fore tlie situations 
^ arise which re(jiiire the decisions to Ik» implemented. It involves 
^deliberate anticipation of situations which are ex^^^cted to arise 
and an effort to bring them under control. \u planning the design, it is 
safest to go right back to the origin of the incpiiry and ask, "Why am I 
doing this particular thing? Will it really tell me what I want to know?" 
After all, it is rather \)oor policy to carry out an experiment without 
having a clear-cut idea in advance of just what is iH'ing tested and without 
an e(}ually clear tentative plan for putting the test into effect. 

The ideal exi)eriment is sometimes descrilx*d as one in which all 
relevant variables are held constant except the one under stuily, the 
effects of which are then observed. But this is not the Ix'st pattern in 
some cases, and in most actual situations encountered in the behavioral 
sciences, it is not practical. Conseijuentiv, some device is needed to help 
correct or «icconnt for the effect of unanticipated or uncontrolled variaWes 
during the course of an ex[>eriment. The introduction of controls provides 
this sort of device. In most cases, the real object of the exi>eriment is 
some sort of comparison, and with the introduction of controls, this takes 
the form of a comparison l>etween the exix'rimental and the control groups. 
But in educational research, it is extremely difficult to secure control 
groups preciselv matched with exi>erimental groups on all relex'ant vari- 
ables. Hence, it is important to haxe at hand a xariety of wavs to provide 
for some sort of control in experimentation. Having a choice among 
several research designs makes it [Possible to select some sensible [^iittern 
of studv which will not sacrifice the verv r(*al ailvantage of controls simpiv 
Ix^cause of the difficulties of securing them. 

The entire design of an ex|>eriment has the function of providing for 
the collection of evidence in such a way that inferences of a relationship 
iK^tween the independent and de|XMidont variables can Ix" ilrawn as surely 
as [Xissible. To achieve this, many research ilesigns have hecn developed 
in the various sciences. VVe will now descrilx and illustrate five designs 
which are particularly usable in studies of teaching and administration. 

DESIGN 1: "KFOtE-AFTEr' STUDY WITH CONTtOL GROUPS 

This design is one form of the classical control group design. It uses 
two groups of individuals which liax^e l>een reasonably matched on variables 
relevant to the experiment or which have been randomly assigned 
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to the two groups. Usiiall) , lx)th groups are measured in relation to 
the dei>endent variable at the In^ginning {Y^) and at the end (Y,) of the 
experimental period. The exix^riniental or indeixnident variable (X) is 
introduced in the experimental group only; it is not introduced in the 
control group. Since the ex|>erimental and control groups are both pre- 
measured, subject to uncontrollable h ctors and vulnerable to unplanned, 
contemporaneous events, the differencvs Ix^tween the sc-ores of the two 
groups should approximate a direct indication of the exi>eriniental vari- 
able's influence alone 

The pre-measures of the experimental group ( V, ) and the control 
group ()' ,) may be used simply to check the initial absence of difference 
between the groups. With tins much assured, the final scores of the two 
groups (Y2 and V^) may l>e compared to analyze the influence of the 
experimental variable (X). Or we may use both the pre-measure scores 
and the i^st-measure scores of both groups to calculate change scores. 
Then we ct)mpare the change in the experimental group (d) with the 
change in the control group {(t). 

Exompto pf a 'Wor^-oftpr" dMi^n with om control group. Samuel 
A. Kirk s Early Education of the Mentalhj Retarded contains an example of 
a "before-after" experiment using one control group.** The experiment 
reported in this book was undertaken to determine the effects of pre school 
education on the social and mental development of young, educable, men- 
tally handicapped children. The reiwt on this experiment has been selected 
for our purposes here because, in addition to illustrating the design, it also 
may illustrate the kind of modification in the "ideal design pattern made 
necessary by restrictions encountered in actual situations. 

To quote directly from the published report of this research study: 
"The difficulties with experimentation in this area are numerous. Many 
factors are beyond the control of an experimenter when he is dealing with 
children, parents, and communities, whose movements and operations are 
their own affair. It cannot be managed as simply or as accurately as experi- 
ments with rats or human experiments using nonsense syllables. One has 
to be satisfied with an approximation of an experimental design. The aim 
of the present project was to set up an ideal exi^erimental design and 
approximate it as closely as possible. 

"Under ideal conditions one would obtain a group of children within 
a specified range of age and mental ability, randomize a control and experi- 
mental group, subject one group to training, and leave the other group at 
home. Identical tests and observations would be made on both groups 
before and after the training period. After five years, comparisons would be 
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made iK'twccn tlu' two groups. \o cliiklrni would jjct skk» no familit'S 
move out of town, and all would attend or not attnid as directed. Since 
tins ideal is ini|)ossil)le. ah approximation of it v. as carried out. and devia- 
tions in terms of loss of eases, randonii/ation. and so fortli. are explained 
in later pages. To olituiu exiK'rimental evidence on tins problem it was 
necessary to set up a project as follows: 

C^omniunity Experiniental Croup. Tliis experimental group con- 
sisted of 2S prescli(M)l nientallv handicapped children from the community 
who were given an enriched nur.sery .scIhh)I environment from nine to three 
oclock each day. At aliout six years of age these children were placed in 
a regular first grade or a special class. There were alxiut 15 children at 
any one time in the nursery scIuk)!. 

"2. (^onminnity (Contrast Group. This group of children consisted of 
26 young mentally handicapiH*d children found in the .same community 
or in neighlK)ring comnnmities. Thev were examined at the .same intervals 
as those in the ex|x*rimental group, hut they did not attend a preschool. 
The term contrast ^roup is u.sed here instead of control group, since these 
children and the children in the Community Exix*rimental Group were not 
randomi/ed in the usual sense. It was im|)ossil)lo to find sufficient children 
at any one time to randonii/.e them." ' 

In this stndN, Kirk also used the same design to conduct a parallel 
study of feebleminded children who had lu'cn committed to state institu- 
tions. This concurrent, but separate, study was set up as a replication of 
the conmiunity pre-sehool exjK*rinient. The two institutional groups being 
compared were named the Institution Exi>erimental Croup (15 children) 
and the In.stitution Omtra.st Croup (12 children). The purjxi.se of con- 
ducting this concurrent replication was twofold. It served as a check on 
procedures employed in the community ex|>eriment, and it revealed infor- 
mation concerning the effects of specialized training on the development of 
institntionali/.ed children. 

The control group-experimental group pattern is admirably suited to 
ex|X*rimental studies; in fact, its basic design elements are essential in some 
form if we are to have a firm basis for infiTences alx)ut the effect of the 
ex|X»rimental variable. But the lull design is lx?set with difficulties in prac- 
tical situations. The investigator should Ix* aware of the jx>tential difficulties 
if he is to make allowances for them, as was done in the Kirk study. 

AdvontogM ond limifaHons. As we have pointed out, this design 
allows us to take account of the effects of initial measurement, uncontrol- 
lable factors, and contem|X>raneous events in determining the influence of 
the experimental variable. The use of a control group allows us to avoid 
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confusing exi>erim«»ntal results with irrelevuncies. We can relate the two 
groups to each other, control or nianipniate the relevant variahles, and 
compare the experimental variable group with the nonexperiniental variable 
group. If the two groups have been satisfactorily matched l>efore intro- 
duction of the experimental (indejiendent) variable to the experimental 
group, and if ade(juate control is achieved to keep experimental conditions 
as pure as [possible, then the difference l>etween the scores of the two 
groups should constitute a measure of the effectiveness of the experimental 
variable. 

However, the measuring process itself may affect the characteristic 
l)eing measured in any typv of research which uses human subjects. And 
the "before-after" design is esi>eciallv vulnerable to this influence. It is 
important to repeat that if j^eople feel they are l>eing used as guinea pii^s, 
or if they feel they are Iwing tested and must make a good impression, or if 
they feel "different" from other i>eople In-cause they are dubbed "exix'ri- 
mental," or if they try to keep their sc*cond responses consistent with their 
first responses or try to vary them if tluy feel that \ariation is expected, 
distortion is apt to creep into the results. The "before" measuring may 
crystallize attitudes related to the experimental variable. The "after" 
measuring may yield contaminated results if the subjects reflect test fatigue 
or l)ecome lx)red or defensive. The investigator who is sensitive to these 
possible influences may be able to minimize their effects through careful 
planning. He may contrive to c*ollect his data as part of a normal course 
of events and not in a special test situation. He may find ways to keep 
pre-measnres and |x)st-measures separate from each other and not con- 
nected in the subjects* minds. He may deliberately avoid overtesting to 
prevent wearing out the good will of his subjects. And he may keep situa- 
tions free of threat to avoid stirring up defensive reactions. 

ExtMftiont of th« control group dosigti. To provide additional safe- 
guards against the influence of pre-measurement, contemporaneous events, 
or developmental changes, more complicated extensions of the control group 
design have been worked out. These extensions involve adding additional 
control groups, each of which is handled differently. These more compli- 
cated extensions of the classical control group design are seldom used. This 
is quite understandable because they are obviously laborious and highly 
difficult to contrive. But a quick look at what we might be doing if we 
attempted to provide for "ideal" controls should illustrate the necessity for 
securing the best control group possible, even if it turns out to be some- 
thing considerably less than "ideal." 

The first extension involves the use of two control groups. The second 
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(M)ntrol group is not |)ir nirasiirrd l)iit is e\i>osrd to tin* iiuiopciulrnt 
(cxiKTiinriital ) variable and tlio |X)st-nu*asnrr. Tlir first i-nntrol group 
is Ixith prr-nu'asiirfd anil iX)st-nuMsnrrd but is not ox|>osod to the ox- 
l)erimental variable. The pre-nieasure for the second control group is 
assumed to Ik* similar to thi' average pre-test score for the exiHTiniental 
group and the first control group. This produces a second control group, 
with an assumed pre-measure and a |>ost-measure, which has Ihvu ex|X>sed 
to the exiHTimental variable but which in no way can reflect tlu* influtMicr 
of the prc-measiirement. If the diffeii nee Ix'tweiMi tlu ^kjsI »sMr* 
scores of the experimental group and ♦ k li thr two contr^J , . 
widely divergent, it mav In* as^nnu'd tin* tti»^- • iMtrracl»»«i '<>ifwr^<i 
the prc-measiir<» and the expt i nnt ntal \ trwAl^ >*»t l» tMcr^H-tMi \m 
found bv comparing the cImanci in s4'ort»N «til fH<- H>rrr grou{>s. Ot course, 
the three groups should Ik* selectetl by random assignment or by some 
nmtching prcK^ediire to insure that they differ onl\ by chance at thr }m» 
ginning of the ex|>eriment. 

The si*cond extension invoKf% ilk* mm* of three control gf^wips. Here 
again, all four groups (one ex|K'rnn<»iital and three control groups) slntuk] 
Ik» selected hi such a way that they differ only by chance in the initial 
stages of the experiment. The exi>erimental and the first control group are 
pre-measured. The second and third control groups are not pre-measnred, 
and their pre-test score is assumed to be similar to the average pre-test 
score of the ex|x*rimental group and the first control group combined. The 
experimental variable is introdnced in the experimental group and in tbe 
second control group. It is not introduced in the first and third control 
groups. All four groups are subjected to a post-measurement. 

Change in the first control group reflects the effects of pre-measure- 
ment, contemporaneous events^ and developmental changes. Change in the 
second control group reflects the effects of the experimental variable and 
of contemporaneous events and developmental changes. Change in the 
third control group can Ik? subtracted from change in the second control 
group to gain an approximation of the effect of the experimental variable 
alone. This can be done because control groups two and three are similarly 
influenced (presence of contemporaneous events and maturational proc- 
esses) with the exception of the experimental variable introduced to group 
two. Thus, the extent to which the change in the experimental group re- 
flects the influence of pre-measurement, contemporaneous events, develop- 
mental changes, or the experimental variable can be determined through 
comparison of the changes in this group with those in the other three 
groups. 
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All of the /n iltiplf control group th^signs really ainoimt to conducting 
sinmltani^Misly the ^^anie exiHTiment two or more times. In essence, this 
amonnts to two or n^ore replications of the same exi^-riment under different 
conditions. If the results nf these experiments are siniilar, we have greater 
♦^*ir;HK'e tliat th(» ontc mum iM>t confounded. If the results are not similar, 
w« prolK' for f.M tors which may have influenced them. You will recall 
Mi.it kirk ct)nducted two simultaneous ex|HTiments (conmnmitv and insti- 
Nifiofi '\|MTinu»ntal and contrast i^nMips) in his study. One experiment 
A as »Hit><i .V a re plication of the other This check-countercheck procedure 
IS whi«t IS ^( n^pted in th<- use <)f iiMttiple control groups 

Tlie nl nuiltiple control gi^nips, and sometimes of even one con- 
trol group, confronts the inxestigator with truly momentous difficulties. 
But Ixlievahle experimental research has not yet discovered ways to dis- 
pense with the c-ontrol group appnmh, difficult as it uvw be. Instead, 
efforts ha\ e heen directed tow^ iM%eiiting designs whkh resemble and 
may snhstitnte for the sometimr* Impossible" classical experimental group- 
control group design. The next four designs may be viewed as examples 
of the classical design, mmlified in ways to surmount anticipated obstacles 
and to make otherwise "mute" question-asking palpable. 

DESIGN 2: "AFTEH^LY" STUDY WITH CONTIOl GIOUP 

Although Design 1 is the most frequently used and approved type of 
design for experimental research, the post-test only design not only avoids 
the possible contamination of pre-test influence and permits research in- 
vestigation in instances where pre testing is not possible, but also provides 
for all of the essential characteristics of true experimental designs. 

While the experimental and control groups are not pre-measured, 
randomization of assignment to the two groups is employed to assure lack 
of initial biases between the groups. Properly handled, randomization can 
suffice without pre-test to lend credence to the assumption that the experi- 
mental and control groups are "equaF before the experimental treatment. 
The experimental or independent variable (X) is introduced in the experi* 
mental group only, and not in the control group. Since the experimenUl 
and control groups are both randomized instead of matched, subject to 
uncontrollable factors and vulnerable to unplanned, contemporaneous 
events, the differences between the post-measure scores of the two groups 
should approximate a direct indication of the experimental variable's in- 
fluence alone. The post-test scores of the two groups (Ya and Y'a) may be 
compared to analyze the influence of the experimental variable (X). 
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In ;ulilition t<» s;itisfvinm tlir Ikimc nMinirt innits of r\prriinnit;il dcsijin. 
tlu* |)ost-t(st uiiK drsii^n iiiiikrs it |M»ssil)lr t(i coiuliut rtsrarcli projects 
ill tliosr (•iri iiinst;iiio('s wIuti' |)n'-trNtiiii» is iiuuK isiililr or iinjKissihlr. 
Ki'Si'iiidi stiiilivs wliicli nnploy iis snl»jrt ts \rry yoiini^ t liiklrni taki n from 
prr-sc'lnH)l or the priiiuirv ijriKlrs frrqiuiitly ciiniiot iisi' pro-trsts wliicli 
riujiiirr fatil'ty witli reading and writing. Also, tlic agrs of rliildrni place 
ratlicr severe limits on tlie leiigtli and complexity of tests wliicli require 
even a ininimnm of sustained effort to einplny language skills. And research 
studies wliicli emplov as subjects older and more sophisticated children, 
or ev en adults, are at a loss to pre test when completely new and iinfainiliar 
concepts tir abilities are the siihjeet of investigation. The numerous current 
studies Inniig addressed to research on the newer media of instruction such 
as educational television, language lalniratorics. programed instruction, 
and the like really have no satisfactory pre tests available. Then wo frc- 
ipientlv are faced with variables alK)iit which we are not free to make 
experimental changes. Relevant variables snch as the father's (Kdipation, 
parents' ediicatifin, and the family's previous places of residence cannot 
verv well Ik* cxperiinentallv manipulated or conditioned by a researcher. 
Sometimes the most im|X)rtant comlitiotiing ex|K'rieiices and moasnre- 
nients have already taken place and further pre-ineasiires might Ik* con- 
sidered indelicate or unnecessary. 

In the case of the ' after-only ' with control group design, we really 
know ver\ little alnnit the * In^fore" phase. But it is possible to reconstruct 
it historically or to conceptnali/e it on the basis of available evidence. 
For instarce, we may use existing IQ scwes, sc1hk)1 marks, ago, sex, or any 
one of nnmorons existing measures, and this generally is done to provide 
more complete evidence. However, the basic comparison is a statistical 
one wherein wo use various correlational and significance of difFerences 
teclini(iues to elimiuato or confirm one factor after another. Or we may 
cIkmiso to control the oxiK*rinient statistically through the use of analysis 
of variance or analysis of covariant*e tochnicpies. 

ExompU of on "crftor-only" with control group dosign. An illus- 
tration of this type of design may Ik* found in the Juno-July 1962 issue of 
The Journal of Educational Research in which John D, Kriimboltz :md 
Barbara Bonawitz report on The Effect of Receiving the Confirming 
Response in Context in Programmed Material.""* This study obviously 
was a case within which no pre-test measures would have been possible. 

The experiment was conducted comparing two approaches of pre- 
senting the confirming response in a programed textbook designed to teach 
prospective teachers how to write valid classroom achievement tests. The 
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"isolation" approach consisted of presenting the desired response to the 
stimiihis frame as a single word or phrase in the traditional programed 
manner. The "context" approach consisted of presenting the confinning 
response as a complete thought, usually hy inserting the desired response 
in a repetition of the relevant part of the stimuhis frame. A control group 
receix ed a completely different program. 

Thirty-two subjects were randomly assigned to the three treatment 
groups. The subjects for the experiment were undergraduates in a course 
in introductory educational psychology at Michigan State Unixersitv. 
Subjects were assigned at random to three treatment groups: the "context* 
group, the "isolation" group, and a control group wliich receix ed a com- 
pletely different program on interpreting test results. Randomization was 
accomplished by arranging the three types of programs in a random 
sequence and distributing them to the students in the order in which they 
were seated in the classroom. 

It was evident that the basic program on writing valid test items 
had a desirable impact on those taking it, regardless of the method of 
icceiving the confirming answer. The control group scored far lower on 
the total criterion test than did either of the other groups. A finding of 
much interest was that the confinning answer "in context" seems to have 
a desirable effect on the ability to respond correctly to application questions. 

A^CNitogM and limilatiom» Although this design shares a problem 
encountered in all research with people (that the measurement proce- 
dures themselves may alter the characteristics they are supposed to meas- 
ure) the problem is vastly less serious here than in "before-after'' studies. 
The assumption is made that both groups are subject to similar matura- 
tional processes and to the same uncontrollable external events between 
the time of selection and the time at which the measure on the dependent 
variable (Ys) is taken. If this assumption is justified, the position of the 
two groups on Y2 at the ctose of the experiment includes events and 
processes that have affected both groups. The difference between Y2 and 
Y'2 may be taken as an indication of the experimental treatment. 

Another advantage of this design for studies conducted in schools has 
already been noted but should be stated again because of its importance. 
A large number of our problems and questions related to schools have 
antecedents which are beyond our control and which alreadv exist. Edu- 
cation is bound to family and community factors which come with the 
child to his school and to his learning activities. Principals and teachers 
can do little, if anything, to alter the father s employment stotus, income, 
or educational level, or the mother s empbyment outside of the home. 



Tims llH*tie 4ik1 iA\wt u^ailuhk* imM%iirc's In* iis«*il in lic*fi (»f -It^sls. 
and llir effect of IIm* c*\|i«TiiiN*iil.il \.irMl»l«\ wlikli ur vun iiMiii|>«il.itfv 
may he invt*»ligalcHl. 

Ttie majiir liniilaliciti <if ilfsi^n is. (if c^iNirv. iIn* |>r(»l>l«*iii of iIm- 
miM(iiif( **tief<ire'ft.*' I>i*%i)(ii 2 fflK .« in^*iI Un liaiKlliiii; ll iliffk-till «»r iiii- 
poiftilJe "lirfcirc*'* fiM*4MirMit(. uinI in .iclclili«Mi is .ipproiiri.ilr in .ill imm s 
wlirre Drtigii I niiglil In* iiv-tl. 

OWON 9: "liFOM AHW STUDY WITN A SMOU OtOUF 

In niany practical .* «iatiiNu. RimIimk a oMitnil K'tNip iim> |>ruM* tn Im* 
not iNily diActilt Init a'M> Mi! iiiiiMrativeK iiii|MM»il»kv Tliift tliircl ilc*%i|{ii 
Ketft around the pnihlrtn «|iiit<* iM*atl\ In liaving (*ach Mil*^*ti m*t\v a« \%%% 
own contnil. The difference lietwecfi the pre-meaMire on \\w ckp«*iMb»fit 
variable ( Vi ) and the pntt-mniMire on tlie dependent %arial)l«* ( V. ) i\ takm 
at a ineaaufr ol the effect of tlit* independent variable ( X ). 

The rin glr ipvMip "liefafr-afler** experiment i« f«mii*timet c'alktl tin* 
^mmscmkmJT etperiment and is often iited in lahorator\' reteanli. \o 
t ec h w i i Mir ia mote common in the total arra> of revrarcli proceduret tlian 
the hr4ofe*and-afler meaanrement of a variable to tett tlie effect <if a 
atim t i hu . an event, or a change which haa been introduced liH%rem tlie 
Ibit and ticond meaanmnenta. In the inatnictioiial iettini(. the ranne of 
chanf^ p yo dm ii H facton which are the auhfeci of itiidv h \er> broad, 
indeed. Adminitlralori and teachen c«an emib and natiiralK manipiible 
tlieie facfora aa e npe rim e nto l variable*, as in tlie sIimK of the ellecl of special 
comTete materials on leami n n ret«-ntion in arithmetic*. At times, lo 
teat the lasting effect of a c^MMiy. more tliaii two measurements are taken. 
The measi nr m s n is air snaced cnvr a cimtimiin|[ period rather than cm either 
fide of an eapetimenlal variable, and the intent bi come s truK sticres- 
sional." Other variaCiom nf this tort of nmnin|[ experiment, of conne. 
are avaflaUa. 

the best Imonm eaample of a sln|^e gnmp ^n^oee^afler experiment asa\' 
be fomid in tbe bnok JUannymmf mi ffte WarA^r * This svas the famous 
study carried out by Mayo. Witith lialwipr. Dichton. and others at the 
llawthome plant of Weslani KleHrir Company in Cliicagn. It was one 
of several so cnllad nMimiraionn stiidira t omrimi lo the social resiae ih of 
the thirtlea. 

Tbe invealigalon had Inltialb come lo the peedic^ton thai the intnv 
duetlon of imprnvemeuls in tbe pli> m<hiI nmditions of work would r e sult 
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in iiKTeased prodiicti%tt\ of an experimental groiip (Migaged hi flu* pro- 
duction of tek|:iioiM> retay as^siemblies. Tlie **bc'forr** |K*ri(Kl c*onsistod of 
intrnsive observation in which production and behavioral normal were 
identified—variation in output, interruptions^* conuniniication patterns, pro- 
c'ion rate» percentat^e of telephone relays reji-cted l)ei*aiiM* of p(K>r 
|iiahty, and m furdi. At subsecpienl interval, tin* variables to In* rnvds- 
uied were introduced- The "after** phaM* consisted of nieasnring the dif- 
ferences in production and in tlie behavioral norms which, interestingly 
enough, impressively followed tin* original prediction. 

But the most interesting finding was what has comv to be known 
among researcliers in tlie beliavioral sciences as tin- "Hawthorne eiUTt.** 
In order to manipulate more exactly tlie physical factors affecting pro- 
duction, tlie experimenters liad set up a special experimental room for tiM- 
girb who were wiring the telephone relays. This wiring room was S4*p- 
arated from the rest of the factor)- and, as a result, tlw* workers bei*anM* tlie 
center of attention. Furthermore, the)' recei%'ed special attention from tin* 
outside experimenters and from the management of tlH* plant. Careful 
stiidiet of this wiring group showed marked increases in production which 
were related only to the special social position and social treatment tin*) 
received. The fact that these workers were made tlie center of attention 
thwarted the intent of the researchers to "control** tlie important variables, 
and almost any sort of change introduced into the conditions of work 
resulted in increased output and impimed behavior. 

The major trouble with the Hawthorne study wskf the absc^tice of a 
control group for comparison. It was possible to assume that liefore tin- 
rek*vant variables were introduced, the eiperimental group was tin* sanM' 
as the control. But tliere was no way to tell what might lia%e liappened 
to the control giuup if there liad been no introductkm of the selected 
variables. Of course, the experimental group in the Hawthorne study was 
placed in a special posttion which dlfferrd from customar) work conditions, 
and this proliably set the subjects apart in their own minds. Sdiool chil- 
dren, acinal oii>d to the Introductkm of dif ering instmctkmal procedures 
and the s nh s e y isn t testing of results, may he somewhat immune to the 
danfer of being mnde special. provkM that the general eoiiditkimi of their 
schoolwork are not suddenly and drastically altered. Nev ertlieless, akhougli 
W9 may sometimes be unable to contrive a separate control group and 
"measure the f#«ct of the non-stlmulus,'* we must at least be aware of the 
dangers in herent in thki gap. 

ArfwMfti mmi IMlMlMt. One of the ma^ir advantages of this 
design is its sheer co nv enienc e and feasibility. If researdi studies of local 
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school problems depended ultimately on separate control and exi>erimontal 
groups, vastly fewer studies would ever l)e attempted. And at the %'ery 
least, it is better for questions to be asked and answers to be sought 
syitematically than for nothing to be attempted l)ecausc* of formidable 
impedimenta luch as finding ''matched'* control groups or locating a suf- 
ficient number of subjects to make possible a random assignment to control 
and experimental groups. Tlie ordinar\' classroom group is composed of 
twenty to thirty students, and this may be the total number of subjects a 
teacher can liave at his disposal. The invention of a design pattern where 
each subject acts as his own control in effect doubles the numl)er of subjec*ts 
available, and no more precise matching is possible. 

The use of this design may be justified wlien tlie investigator has good 
reason to believe 1 ) that the ''before'* measure will not in some way affect 
the response to the experimental treatment, and 2) that there are not 
likely to be any other influences, betides tlie experimental treatment, during 
the course of the experiment which might affect the subjects' responses at 
the time of the ''after" measurement. In order to be reasonably sure tliat 
such assumptions are justified, the investigator must have considerable 
knowledge of the probable effects of his measurements and of conditkms 
other than the experimental treatmeit that ace likely to influence the 
dependent variable. This is fortunately true in tlie case of individual scliool 
environmenls where principak and teachers are apt to approach problems 
with an already impreisive array of knowledge about tlie subjects and 
where they normally exercise considerable control over tlie planning and 
conduct of the learning environment. 

Another advantage can be found in small V studies ( 25 or so sulijects ) . 
In contrast to large-scale experiments, the smaller number of subjects 
makes possible deeper and more complete study of individiuil subjects. 
In addition to analyses of a group of subjects-for example, statistical 
analyses-individual case studies can lie completed. Kirk did this effectively 
in his study. In truth, the staff members of individual schools an* in a 
unique position to exercise a kind of control and cxmipleteness tliat is 
simply not available to outside researchers. 

Some t i m es, the "hefore-after"* design used with a single group of 
subjects is criticized because it is alleged tliat the findings cannot lie 
generaliaed to other groups. In the case of the scImioIs, \\\\% may not 
strictly be the case. The teacher-researcher wants to engage in imesliK^- 
tkms that wifl help him make better decisions and improve his prac^iies 
note;. He also hopes tluit his flndinp will apply to tlie childn^n Im* will 
teach in the fiitiire-prohably in the same scliool and at tlie same grade 
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level Whether they will apply, of course, depends upon tin* extent to 
which his students tiotv represi^nt a random sample of tlie ix)pnlation lie 
will have in the future. 

Stephen M. Corey, in his hook Action Research to Improve Sc/iih>/ 
PracticeM, presents some interesting material on this point.'*' He reports 
on a series of sUti^tical tests of tlie null hypothesis tliat tlie distrilnition of 
scores on individ .al classes and the distribution for tlie total poi>ulation 
of N classes taught by t\ie same teacher will not differ more tlian niiglit 
l>e expected as a consequence of chance. Students in twelve different class 
sequences extending over three years, taught by twehe teacliers in Ne\\ 
York City and West Orange, New Jersey, were the subjects of tlie analysis. 
The variables studied were IQ, subject matter achievement test scorers, 
and silent reading test scores. The chi Mpiare test of tlie significanc*e of 
differences was applied to the distribution of scores. The probabilities 
resulting from thirty-eight clii square tests, with one possible exception, 
indicated that the students in a teacher's class at any one time could be 
considered to be a random sample of the total population of several 
consecutive classes. Of course, any teacher, if he lias avaibUe the 
necessary scores for several consecutive classes, can appl> tlie chi square 
test to determine if the differences between tht* classes are greater than 
would be eipeeted by chance ulme. 

It is quite pertinent to inquire into how much confidence can be 
placed in the generalizations derived from a single study, even if the 
study s sample was randomly selected from an indefinitely large popula- 
tkm. For example, some investigator miglit use a random sample of all 
fifth graders in the State of New York or the State of California. But teachers 
anywhere wouM do well to question whether the immediate group with 
which they are concerned is at all similar to the one on which the ""ran- 
domly sampled"* study was based. Research studies in educatkm hrve kmg 
been criticised for the tendetic)' of researcliers to generalize on the slim 
basis of a single study completed once and never again. There is a strange 
lack of replication of studies in ediK*ation, to the detriment of believable 
scientific knowledge. In this sUte of affain. it miglit be wise for any 
practitioner in educatkm to form tlie liabit of performing dieck studies in 
his own situation before accepting gi*nerali/atkms derived from unrepli* 
cated studies done elsewhere. 

Perhaps the ma)or limitotkms of this design liave lieeti adecpuitely noted 
in relatkm to the liawtliome effect. The chief weakness is, of course 
that there is no control group to provide infcNrmation cm the effect of the 
nonstimukis. 
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MSION 4: ''Afm-ONIY'' STUDY WITNOUT CONTIOU 

Perhaps a design equally common with llie one ]u%t deficril^ed is tlie 
single cell. Tliis generally rq>resc»nU Hie over-all desiign used in scliool 
surveys, many research studies, assessment studies, status studies, and case 
studies. Treatment of tl»e data is usually descriptive in nature, with only 
incidental attempts to attribute findings t<i associated variables. As such, 
this is actually a report on wliat exists at the time of tlie study. It is not 
a design used in experimental research, tlioiigli it niav Ik- a necessary step 
in cluirting tlie territory for later experimentation. 

When we luive gatliered adecpiate data aliout tin* group lieing studied, 
we can rebte various cliaracteristics and gain information aliout factors 
which may have been previously obscure. Hie more we stud> such a single 
cell, the more confident we become tliat certain forees may be influential 
in sliaping the results we uncover. Our logical analysers may jibe with 
good common sense and suggest a flood of h>potlieses to be tested later, 
Stiidiei conducted within this design can be very useful. Imt they do not 
test reasoned sohitions to conceptualized problems. Ordinarily, surveys and 
sUtus studies are inMifilcient for experimenUl puqwses. The chain of 
events associated with experimental research calls for some sort of both 
control and experimenUl groups to see 1 ) how the experimenUl variable 
has afected the experimenUl group, and 2) how tlie lack of, or difference 
in, the experimental variable has affected the control group. 

iMmpto M '^dhm^t^t^ alv^ wMimI MUlrala. One massive 
"afUr-oniy" study was conceived and conducted by the twenty-seven 
members of an ASCD Yearbook Committee and published as the 19S8 
ASCD yearbook. A Loo* 0t ContinuUij in the School Progmm V Tlie com- 
mittee members choae to study the present situatkm with respect to articu- 
lation and learnln| continuity as viewed by children from kindergarten 
throi^ the senior high ichool. To do this, the commiHee members worked 
out a simple Interview guide lo give a structure children s reports in a 
group interview situation. Committee memhen or their represenUtives 
talwd with groups of children about experiences which had helped or 
hindered them in their progress through school. Then they asked the 
individual children In each group to answer four simple questions written 
on a m i m eogr ap hed guide sheH: 

"1. Ten about anything that has happened to you which has helped 
you to feel that your pvogrosa was smooth and that the school helped you 
move •long wtthout unmcetMry diflcuky. 

"2. TcU about anythinf that hat happened to ymi which has nwde It 
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clifliciilt for yon to move along siiiootlilx through sch<K>l. 

"3. Toll alK>Mt any (*\pcTi( iicr that has ]yovu \ vt\ pleasant or \ rr\ 
implt asant and which gn w out of this <|ut stion of yonr progress throngli 
scluK)l It max have hap{MMu>d (piite a while ago or latelv. Write as nnicli 
as yon can renienil>er alxxit it; tell Ik>w it made yon feel at the time it 
hap|HMied and liow you feel about it now. 

"4. If yon have movcxi from one scliool to another, tell how \on felt 
alK)nt moving, before and after you moxed. Do \ou fec»l the same wax 
iK)w':^'*^* 

The cYmnnitttH^s plan for gathering ilata pnr|)os(*ly allowed for \aria- 
tion in res|)onsea^. Each wmunittee niemlxT was left free to modifv tlu» 
interview guide form in au\ way that seenieil desirable to him. as long as 
lie got children to give tlHMf own frtv re|>orts to the four Imsic (|nestions. 
Particularly with yotmg children, tin* interxiewcrs had to modify and 
reword the questions. In some cases, it was necessiiry to take notes, use 
recording devices, or arrange for stenographic reports. 

Sets of student reports were submitted by .veventcH^u committc*c* mem- 
bers from eleven states. They liad collected a total of 2731 usable reports. 
A group of analysts ( tin- committee chairman and twelve advaiici*d students 
of the University of Alabama) reviewed the c^pkied reports and worked 
out data sheets to which tlie material could be transferred. Tliis procedure 
resulted in organizing the data under r-ommon categoric>s regardk^ss of 
where in a child s report tlie peHinent terial was recorded. An\ tvpe of 
situation which represented at least one pet cent of tlie total situations was 
included as a category in this grouping. TIr» data slieets were also classified 
according to grade fc»vel; primary, intermediate, junior high scliool, senkir 
high scliool, and a mixed group for wlikrii tlie grade k \el at tlie time of 
tlie incident was not indkrated. A total of 4197 data sheets were used in 
tlie final summarizatkNi. 

The completed study organized the data by grade k-vel groups around 
fourteen categories of situatknis. Eleven tables were revpiired to present 
the summarized data. Tlic data in tlie tabkni were grouped acixirding 
to frequencies, percentages, and rank order of positive and negative 
reactions* 

The fourteen categories of respoiisi*s to the initial four guide slieet 
qiiestimis turned out to be: I) moving to new scliool community, 2) 
teacher behavkNr, 3) subject matter, 4) moving to next school k^ve!, 5) 
smooth progress, 6) extracurrknilar activities, 7) differences in teacliing 
inetlMids, 8) illness, 9) rewards, 1«) punishments, 11) promotion, 12) 
grading, 13) retentkm, and 14) acx^iik^nts. Interestingly enough, the per- 



vcuUxi^c nf iH gativr n sponsrs rr(l<»i t<Ml nii ihc ilata shvris was ijn\it<»r than 
tlir |)4*l'ct'nta^^ of p<isiti\r irsp nisrs. 

Tin* writiTs <if this vrarlnKik wvrv explicit ;nul positive in tlitM'r 
insistcMici* tliat tlir stiiily was nnT(»l\ ;m intnKhutory anil <'xpl<>rator\- 
treatment o\ a sniijcx't wliicli is in uvvd of mncli stiiily, research, and 
evahiation. 

AdvontogM ond limifotiom. The snr\e\ nr iU*scripti\e stiuly is a 
pr(K*ess for learnint^ (HTtinc^nt information alMint an existing situation. The 
principal devic(»s h>r gathering data from iH'ople involved are the inter- 
\ie\v, the (piestionnaire. and sinnniaries of existing d(K'iinu*nts which coinit 
and emmieratc plu*nonu*na. such as those which arc published by the 
Hnrean of tin* Census and by the Hesearch Division of the National 
Kdncation Association. 

The survey frecjnently lK*conu*s more than a mere fact-finding device. 
It niav re sult in important hyix>thes( s or conclusions that help to solve 
current problems, and it may provide basic information for comparison 
studies and for identifying trends. Surveys completed five or ten years 
ago nia\ Ih* re|K*at(Hl, and something will Im* learned alxnit changes which 
have taken place in the meantime. ScImkjI district surv(»ys have long serxed 
certain needs in education by prov iding previ«nisly lacking descriptions of 
curricula, attendanct* centers, citi/en opinion, and physical facilities. They 
have also hel|H*d to \xh}\ divergent ideas, tc*chni(pu*s, and bits of informa- 
tion, thus throwing light ni)on (*xisting conditions in neiul of change and 
improvement. 

Most times, it is far easier to obtain data through single cell studies 
than it is to draw valid conclusions from tlu» facts discovered. Studi(»s of 
this tv|>e tend to ha\e a larger sco|M' and to rangt* more freely than experi- 
mental studies with control groups. Surveys fre«|Uently t(»nd to Ih» com- 
jN)S(*d of a kK>se confederation of M»\<»ral single cells related or unrelated 
to each other. These eharacteristies lend to studies the impressions 
of si/e and convincingness. And hen^in li(*s ix^haps tlu* major limitation 
of single cell studies. The fretnlom to range more widt^ly and not to Ih^ 
strictiv liound by the r(*<|uirements of ex|)(*rimrntal re,s<»arch also makes 
ini|M)ssibh* definite problem solution, prediction Inised on tln*ory. and 
prolKibilttv inference. In spite of feelings of certainty which ma> Ih» 
drawn frcmi singh* evil studies, it is im|)ortant to note that such fi*(*lings 
or mnvictions are m>t d(*moustrations. The \'vr\ real and necessary ttm- 
trilnitions to Im» made through pre-res<»arch studies. eas<» sludit s. surveys, 
status studies, and other forms of enunierative investigations should not 
hv diluted by reading into tln-m niore than can Im- demonstrated. 



17 



Hrwarrh fnr the rrtictitumrr in Education 



DESIGN S: EX POST FACTO EXPERIMENT 

The basic pattern of the classical exiHMiinental design is one of com- 
paring the result of the experimental variable iiix)n an exixM imental group 
with the result of not introducing that variable into a comparable group 
under the same conditions. Most frecpiently, this involves a prediction of 
what will Ik' the difference in results Ix^tween the two groups, i.e., it is 
used as a projective design which looks forward in time. However, the 
chissical logical pattern need not Ik* confined to projwtive designs. It can 
also Ih' applied to problems which look from the past to the present rather 
than IxMUg oriented toward the future. This is known as ex ix)st facto 
research. 

The major characteristic of this procedure is that the inx estigator can 
control the crucial variables only by selecting those which have already 
iK'en recorded. The investigator may k^cate a group of subjects which has 
l)cen exix)sed to the t>pe of exi)erience in which he is interested and 
another group, similar in other relevant resfiects, which has not been 
exix)sed to such an exix?rience. He then compares the groups on the basis 
of present {Performance. In basic logic, however, projective and ex post 
facto experiments are the same. The purpose of both is to compare two 
groups similar on all relevant characteristics save one in order to measure 
the effect of that characteristic. 

In the ex post facto experiment, we manipulate pieces of paper instead 
of experimental conditions. Tliat is, we manipnipte existing records which 
symbolize the behavior upon which the experiment is focused. Control 
is achieved by matching records in such a way that the control and expert- 
mental groups are similar in relation to all hnt tlie crucial variable. The 
records are then followed through by tsuring the consiH|uences as 
obserxed in present behavior. 

Tlie teacher-investigator may not be in a position to test a hypotliesis 
by assigning subjects to different conditions in which he directly controls 
the experimental variable. For instance, there very well may be a limit 
to which certain types of instruction can be justifiably withheld from a 
group of control students. Or tlie teacher may be interested in exploring 
the effect of some long-range scIkx)1 practice but simply does not have 
time to wait several years for matnrational effects to become evident. 
Consider, for example, tlie research that has been done on incidental 
(activity type) instruction in arithmetic. The extent of incidental instruc- 
tion compared with formal instruction has not been controlled, as it would 
be in a "purely** experimental study, by assigning different individuals to 



Desigm and Decixiont 



classroom groups where instruction proceeds according to different basic 
curriculum patterns. Ratlier, records have been secured wliich symbolize 
the arithmetical behavior of students in activity (experimental) schools 
and students in formal (control) scliools. The relation between type of 
instruction and performance in arithmetic is then computed. And for many 
reasons^ it is not likely that the basic design of a school curriculum will be 
manipulated to suit experimental conditions. 

The ex post facto pattern involves the same problems of matching that 
are present in projective designs. However, it has two additional problems 
that are uniquely inherent: 1) It is necessary to find adec]uate records 
after a lapse of some years; and 2 ) Problems are restricted to those areas 
for which adecinate records exist and are available to the investigator. If 
important material was not recorded, the needed information cannot be 
resurrected just for the purpose of study. 

However, teachers find themselves in an unusually advantageous posi- 
tion to make ready use of complete records that go back in time. Adminis* 
tratively, schools have become avid storers of information: academic 
marks, intelligence quotients, health charts, attendance records, height and 
weight charts, aptitude scores, citizenship marks, parental information, 
judgmental descriptions by teachers, personality scores, achievement scores, 
sociograms, and a host of additional measurements. In fact, there are very 
few other sources of meticuknisly kept records which cover so wide a 
range of the behaviors of people over sustained periods of years. Theo- 
retically, it is possible to reconstruct Johnny at any particular time— in 
terms of his father s occupation, the story of his physical health, his rate 
of physical growth, his intellectual achievements, his sequential progress 
through school, what his teachers thought of him, how he got ak>ng with 
l^laymates, what his talents were, the devebping pattern of his abilities, 
/and many more qualities belonging to johnny and classroom groups around 
/ him at various times. 

Frequently, the investigator may not find it pouible to assign indi- 
viduals to different groups, one of which will have the experimental treat- 
ment and one of which will not. An alternative solution is to find two 
comparable groups which will be, or have been, exposed to experiences 
.which differ in relation to the crucial variable of interest. This alternative 
Us a very common procedure in most types of research studies and serves 
^ery well in the case of ex post facto experiments. 

Ijiwnfte •f M M pmt fMl9 wpJkmM. A particularly ingenious 
scheme was used in one study, The Grouped and the Ungrouped," by 
Jo Taybr Auld.'" This study set out to investigate the effects of traditional 
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"high," "average/' and "low" ability grouping in elementary schools. It 
could not have been conducted at all if there had not been already existing 
records which could be manipulated on an ex post facto basis. 

Two elementary schools in a South Carolina city were included in 
this study. The neighborhoods served by the two schools were alike in 
terms of housing, community facilities, churches, etc. In each school, the 
principal had held the post for more than fifteen years. The number, age, 
and experience of faculty members were approximately the same. The 
facilities in the schools were equal, as were the organized play programs, 
library periods, and so forth. Being in the same city system, the schools 
operated under the same policies and followed the same testing pr^y^edures. 

In School A (the experimental school) there were thirty-one boys and 
twenty-seven girls in the sixth grade who had been in attendance there 
since they entered the first grade. These children had been grouped in 
"high," "average," and *'low" groups when they entered the second grade. 
The grouping had been based on teacher judgment and on reading ability 
as measured by the California Achievement Test. 

In School B (the control school) there were twenty-nine boys and 
twenty-seven girls enrolled in the sixth grade who had been attending the 
school since they entered the first grade. These children had not been 
grouped in any manner throughout their six years in school. In other words, 
they had been in heterogeneous classes throughout their school experience. 

In each school, there was an active PTA with a membership of 100 
per cent of the families. The families were much alike in the two schools 
with regard to their socio-economic status. More fathers in School B were 
engaged in professional occupations, and slightly more nK>thers in School 
A were employed in positions outside the home. 

The median intelligence quotient of the group of children in School A 
was 107, with a range from 87 to !34. In School B, the median was 110, 
with a range from 87 to 133. 

During the 1999-00 session, Auld studied the test results obtained 
when the children in School B were in the first grade. On the basis of the 
reading test and the recorded teacher evaluations, she arranged their 
names in three groups— "high," "average," and "low"— in the same mariner 
that the children would have been grouped for instructional purposes if 
they had been enrolled in School A when they entered second grade. Thus, 
she was able to establish comparable groups in tlie two schools and study 
the differences, if any, that existed in the students after more than four 
years of schooling under different systems of grouping. 

The following findings are based upon the results of the Metropolitan 



Designs and Decisions 



Achievement Test ( Intermediate Battery VI ) which was administered to 
the children of }x)th schools. 

L There were no statistically significant difFertMices in the perform- 
ance of the students in Schools A and B who were in the "high" or "average" 
groups. 

2. Among the "low" group, those in School B demonstrated signifi- 
cantly more achievement than those in School A. This difference was quite 
marked, with tl)e School B students earning a mean grade equivalent of 
6.3 whereas those in School A earned a mean of onlv 5.0. Tins difference 
was significant at the .01 level of confidence. 

3. Achievement in reading, arithmetic, and language on the part of 
•average" and ''low" groups was higher in School B where the students 
had not been grouped for instructional purposes The data indicate that 
at least among these students in these two schools, grouping over approxi- 
mately a four-year period apparently hindered the achievement of the 
average and below average pupils, and it did not provide advantages for 
the above average or "high" pupils. 

AdvQiifogM oml limifoHom. The major advantages of tlie ex post 
facto experiment cluster around the simple fact that it makes it possible 
to seek answers to certain kinds of questions that otherwise would go 
unanswered. Tlie ex post facto pattern in effect compresses time, per- 
mitting study of the effect of many years' experience now, rather than 
waiting for the experience to happen. In a very real sense, this expresses 
a common and regular responsibility of all schools everywhere. Measure- 
ments are made and records are kept to predict the probable success of 
students in college, in jobs, and as citizens. Grade point averages, academic 
experiences, and citizenship marks are used as predictors of future abilities 
and acx^omplishments. These records, if carefully secured and preserved, 
can provide the needed data to check the educational procedures which 
gave rise to them in the first place. 

Of course, the investigator will find it necessary to account for several 
factors which might operate to confound tlie data. He will want to be 
sure that the individuals who have undergone different experiences were 
comparable before they were exposed to the different experiences in 
(juestion. He will want to be sure that the time order and intensity of 
exposure to contrasting variables are similar. He will account for some 
of the extraneous variables which might have operated during the course 
of the experiment. And he will avoid such obviously confotmding factors as 
comparing an assigned group with a self-selected group. The Auld study, 
for example, accounted for factors such as these. But it must be remem- 
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\jcred that her study could not do more than iipproxiuuitr the control group- 
experimental group design. 

Tlie major limitation of the ex |K)st facto exixTimeut is that it cannot 
provide safeguards as adetpiate as those given hy random assignment of 
individuals to experini(*iital ami control groups, control over the extraneous 
variahles that might o|XTiite during the murse of the ex|K?rimeut, and 
direct manipulation of tlic* experimental variahle. In addition, the experi- 
menter is limited by the degree of thoroughness and accuracy which un- 
known and past record k<'«*|X'rs have dispUiyed. And of real concern and 
frustration is the danger of encountering "blank s|X)ts" where some crucial 
(lata may not have* lH*en rcx-orded a^ all or ma\ liavt* In^en recorded in au 
unusable manner. 

In the same sense that the creative investigator may view himself as 
one who observes what hapiH'us when he treats the world us if it would 
operate as he hv|)Otliesi/.es, so he may view himself as one who can plan 
(design) events to facilitate the 0|KTation of his liy|X>theses. He devises 
ways to control and maniiinlate conditions to make observatious of his 
hypotlu*ses in action clearer and sharper. This calls for ingenuity and in- 
vention. It also calls for the recognition of op[x>rtunities inherent in the 
original problem and its liy|>otlieses in order to exploit new ways to plan 
for c^ll(*cting data or to modify ahead) known designs which have lH*en 
successfully used elsewhere. For instance, Auld employed unusual insight 
in employing the ex post facto records in one lieterogeneously grouped 
school to arrange the children's names in the same manner that they would 
have lieen arranged had they lx>en in a homogeneously grouped school. 
In this manner, slie contrived a wa>^ to control and manipulate conditions 
so that her hypothesis could he tested. 

It would be a mistake to regard tde varions designs for conducting 
tests of hypotheses as complete, final, or restrictive. The world is wide 
open for new and thoughtful wa) s to plan the conduct of problem solving. 
And theoretically, at least, there should Ik* as many different ways to plan 
as there are ditFcrent problems to l>e solved and different i>eople to solve 
them. 

This chapter has presented only five types of "standard" design used 
in research. Many more are descril>ed in the literature on research methods. 
Some are highly complicated and include unduly difficult or impossible 
requirements for jvople engaged in actual situations. Others are purely 
statistical, focusing chiefly on mathematical models. But the problem of 
design is mainly logical in nature and is only secondarily statistical for 
the ordinary inx eittigator. School |ieople would do well to expand gradu- 
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Figure 5. Types of Experimental Detignt 



allv their knowledge of the alternatives suggested hv any available design 
pattern and then choose the ones most feasible, promising, and flexible 
for their own research interests. 
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PThe data resulting from any expc*rinient are nsnally a collection 
of ol>servations or nu-asuremiMits. We have H*en that tliese data 
are more likely to yield cn*dihle information and apply to the 
answers that we si^ek when tin* experiment is gnided hy adroit 
hypothesizing, carefnl sampling, and insightful pre-planning or 
designing. In addition, when we considerc^d variahles and probability, 
wc saw that tho testing of hy|)othes<*s nnpiires fonr of evidence: 

1 ) concomitant variation, or evidence that the inde|HMident variable (cause) 
and the de|)endent variable (effect^ are usMxriated; 2) time seqnence, or 
evidence that the de|x*ndent variable did not cKviir liefore the independent 
variable; 3) internal control, or evidence ruling out otlK*r factors as possible 
influences; and 4) probability, or allowance for tlie opcTations of chance 
or sampling error— instead of the delitierblely introduced independent vari* 
able— to have produced the effects that we observe in an actual experiment. 

During and after the collection of data, it is necessary to analyze them 
in terms of tlie four types of evidence which lend reliability to tlie con* 
elusions reached. Logical analyses apply quite handily to the first three 
types of evidence. But the fourth type of evidence, allowing for the 
operations of chance, can very seldom be ascertained by simple direct 
inspection of tlie data or by bgicai analyses of them. Stat!.stics provides 
the nnetlMxk>k)gy whereby this can be done. 

NONPARAiMITmC AND PAtAMITMC MITHOM 

The fundamental statistical problem arises when we try to see if one 
variable produces a change in another variable in a situation where the 
latter MpanianeouMly changes due to the influence of other causal factors 
which are commonly termed ^'chance.** In other words, starting with a 
mathematical model for random variation, we ask what the chances are 
of drawing two separate samples whose mean or median difference is as 
great or greater than the difference obtained from an actual experiment. 
If the chances are small (say one in a hundred) of obtaining a difference 
of this magnitude from two random samples drawn from the same parent 
popubtkNi, we may tend to regard favorably the likelihood that the actual 
difference is due to the effect of the independent variable which was 
deliberately introduced into the. experiment. But if this likelihood is con- 
siderably greater (say fifteen, twenty, or more in a hundred), we dismiss 
the difference as easily due to chance. Of course, this reasoning involves 
the assumption t!uit the variance in the population or samples with which 
we are dealing in an actual experiment is normally distributed, as it is 
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in the mathematical model. If it i^ imt-and it oftc*n is not<-tlu*n * nsti- 
fication for tlie comparison is seriously questionable. 

Because such assumptif>tas are often unsuitable in practice, and be- 
cause tlie recpiisite conditions on the assumption of 'normalev * in tlie 
distribution of sample or population scores either are iMit met or are un- 
testable, increasing attention lias recently been |>uid to the newer statistical 
tests that do not depend upon such assumptions and c^Miditions. These 
are the nonparametric or **distribution-free** tests. Some nonparametric 
techniques are called ' ranking tests" or "order tests/' and theii- titles suggest 
their characteristics. 

Tlie customary **stuodard** techniques of statistical inference are based 
oi. several strong assumptions about tlie nature of tlie population from 
which the scores were drawn. For example, a technique of inference may 
be based on the assumption (seldom tested) that tlie scores were drawn 
from a normally distributed (bell-shaped) population. These population 
values are called "parameters*'-a mathematical term indicating variable!^ 
entering into any distribution where tlie possible values of the variable 
correspond to the distribution. Because i f this* tlie usual statistical tech- 
niques are called parametric. Ttiat is. parametric techniques are tied 
clcMely to the shape of the distribution of scores or obsi>rvations. When 
using parametric tests, we arrive at conclusions which may be valid if the 
as9umption$ regarding the shap of the population are valid. 

Nonparametric or distribution-free techniques require fewer quali- 
fications and assumptions regarding the shape of the population. In test- 
ing hypotheses about a control group and an experimental group, the 
comparison is between distributions and not between parameters. When 
using nonparametric tesU, we arrive at conclusions which may be valid 
regardleee of the $hape of the population udues. 

Several disadvantages and advantages of nonparametric methods have 
been pointed to by Lina>ln E. Moses/^ Paraphrasing the Moses treatment, 
we may compare on several counts distribution-free techniques and normal- 
distribution techniques. This may provide a ratlier quick summary of 
reasons why nonparametric techniques seem reasonably well fitted to re- 
search projects undertaken in individual school systenru, individual scboob, 
and individual daaarooms. 

MMirfvmNigM ai mmmfmmtm^rit moHitrft. 1. Nonparametric tests 
applied to normal data-*rather than normal-theory tests-are wasteful of 
data. The depee of wastefulness is measured by the ''power efRciency** 
of a nonparametric test when compared with a simibr parametric test. 
"Power"* can be defined as the probability of rejecting the null hypothesis 
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If, for i*xanipli\ a icsi Uds 80 ptT cvnt cfficifiicx , tins uwaus that 
where the data are from a normal (Ihtrihution, tlir appropriate* clasitical 
tfst would Ih» just as offwtivf with a ^nipU* of 20 jx^r cnit sinaUrr s\/x\ 
III thiii way. "efficiency' (a mathematical test of statistical tests, lK>tli 
parametric and iionparanietric) expresses the relative* in««rits of the nou- 
paranH*tric test and the classical test under ainditioiis where the nonual 
test is eorrcet, It ilovs not, liowever. tell us how the tests will coin|)are 
on non-normal data. 

2. For large samples, some of the iionparanietric methods require a 
great amount of lalN>r. 

3. Writing in 1952 and 1954, Moses listed as another disadvantage 
the fact that tiK* nonparametric tests and tables of significance vahies were 
widely scattered in the |x-riodical literature. Since 1954 this objection has 
Ikh'ii largely overc*onie through the publication of several quite usable 
iKioks which c^ontain lN)th the methodology and the necessary tables. 
Among these are: 

VVilfri^d J. Dixon and Frank J, Massey, Jr. Introduction to Statistical 
AfUthjsis. New York; Mt<»raw-Hill, 1957. 

Norville M. I>ownie and Roln^rt \V. Heath. Basic Statisti Methods. 
New York: llarjx^r & Brothers, 1959. 

George A. Ferguson. Statistical AnaUfsis in Psychology and Edu- 
cation. New York: McCraw Hill. 1950. 

Sidney Siegel. Nonparametric Statistics f^^ the Behavioral Sciences. 
New York: McGraw-Hill, 1956. (Tliis book, entirely given over to 
nonparametric tests, contains a very large c*ollectioii of appropriate 
tables of significance values.) 

Merle W. Tate and Richard Clelland. Nonparametric and Shortcut 
Statistics. Danville, Illinois: Interstate Printers and Publishers, 1957. 

Tlie appearance of tlie literature on nonparametric statistiet has been 
matclied by the spread of knowledge and the use of distribution-free 
methods. Just a decade or so ago, nonparametric techniques seemed to 
be the exclusive property of nuithematical statisticians in both this country 
and England. Since that time, researcliers in many theoretical and applied 
fields of knowledge— psychok)gy, agriculture, sociology, biology, education 
—have learned to use these newer techniques. 
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AdvontagM of fionp«irom«trk iMfhodt. I. If sampK s an* wrs small 
(i\s small as six or frurr iiumiiIhts ), {hrrv is no altrrnati\(* to a 
I lonpa ramc* t ric test . 

2. If till* sample* consists of ohsiTNations from several diffrrrnt (Kipn- 
lations (ai <l perhaps a diffcn nt nnmln^r of casrs in i\\v\\ snl)samplr). tlu*rr 
ina\' Ik* a suitahU* nonparamcfric tr(*atnu*nt. 

3. In certain cases, data can only In* tak<'n as "In'tter" or "worsi\" 
or "gain'' or "loss." That is. an ohserxation can only Ih* <*\pr<'ssed i\s A 
pins or minus. Ohvionsly, the classical t(*sts arc not applicable to sncli 
data, bnt some distrihntion-frci* tests arc 

4. If the data are inherently in the nature of ranks, not measurements, 
they can Ih* treated directly by nonparametric metlunls without assnming 
some s|x*cial sh«i|K* for the nndcrlving distribution. 

5. Whatever may Ih* the shapt* of the distribution from which tlu' 
sample has bt*en drawn, a uonparamctric test of a s|Hrifj<'d significancf* 
level actually has that exact significance level. 

6. N(m|)iiranietric methcKls are usually much more understandable 
and much easier to apply than the classical tc^clmicpies. Pencil and scratcli 
paper will do (piite ade<piately for most calculations, and there is nnicli 
less need for (*l(*ctronic computers. Possibly this is one* of the strongest 
arguments for thes<* methods in the cast* of inv(*stigators who nnist s|)end 
something less than full time on research ar tivities and without the aid of 
expensive calculators. 

MIASUMMINT SCAUS 

At this point, the nature of the data witli which tlie liehavioral sciences, 
including education. m<i$t deal is very much to tlie jwint. The operations 
allowable on a given set of scores are dependc*nt on the level of measure- 
ment which has been achieved. The tlieory of measurement consists of 
a sr*ries of levels or scales of measurement: I ) nominal. 2) ordinal, 3) in- 
terval, and 4) ratio. 

The fwminal scale consists of numbers or other symbols which are 
used to identify and name groups to wbicli various persons or objects 
belop^;. Identifying boys and girls as B and C (or 1 and 2) is an example 
of nsin\ a nominal scale. 

The ordinal scale assigns numerals to objects which are rank-ordered 
with respect to some characteristic. Th# persons or obfects can be ranked 
starting with the highest score, and then proceeding to tlie next highest, 
and so on until the k)west score is reached. Or the ranking can start with 
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the lowest score, proceed to the next lowest, and so on until the highest 
score is reached. Thus, the r ink of students in a graduating class, or the 
order of students from high to low on exanunation (questions correctly 
answered, gives us an ordinal scale. 

The interval scale defines a unit of measurement such that a ilifferenci" 
of one scale value is e(|ual to uny other one scale value. Dates on a calen- 
dar, degrees on a thermometer, or carefiilh constructed standard achieve- 
ment test scores may approximate an interval scale. 

The ratio scale is an interval scale which starts at an ahsolute zero 
|K>int, so that it is correct to s|K*ak of one scale \ alue as hcing twice, or 
three times, etc., as large as another. Thus, weight ot ohjects expressed as 
|K>unds, or ounces, or grams has a true zero point, and the ratio hetween 
any two |X)ints is independent of the unit of measurement. But IQ scwes, 
for example, are not ratio measnreuients since there is no such quantitv 
as 0 IQ, and an IQ of 120 does not simply indicate* twice as much intelli- 
gence as an IQ of 60. 

The major |X>int to l>e servetl hy this rapid consideration of measure- 
ment scales is that typically in the iK'liavioral sciences, onr measurements 
are unavoidahly crude and inexact— compared with the physical sciences, 
for instance. It may be illustrative to c^insider that ratio scales arc rarely 
achieved in the In^havioral sciences. Thus, research in education is left 
with three tyix's of usable scales: nominal, ordinal, and interval. Non- 
parametric tests are most useful with the first two tv|H's of measurements, 
in the order listed, and to some extent with the third type. Parametric 
tests are most useful with interval scale level of measurements and pro- 
gressively less useful with ordinal and nominal levels of measurement. To 
the extent tlmt the hulk of onr judgments, scores, and measurements in 
education cluster armmd the nominal and ordinal scales, distribution-free 
techniques increase our res<'arch e(|uipment and flexiliility. 

STATINO AND EVALUATMO TNI NUU HYK>THISIS 

Wlien we have secured the desired evidence, as expressed in terms of 
any measurement scale, we then procefHl to evaluate it in regard to levels 
of significance. We will next c^insider what is meant hv ''significance/' and 
liow this level is determiiml in actual situations. 

It has been stressed repeatedly in this publication that tlie job of 
statistical studies is to handle the problem of the nu// hypothesis (the 
hypothesis that the results obtained for the differi ice between two or 
more groups is not due to the delilxTately introduced independent vari- 
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able, but is due to an error or accident ot ^^ampling). Evidence is i cd not 
to "prove" some positive hypothesis (a logical impossibility) but to pro- 
gressively "disprove" or discredit tlie mill hypotlu*sis. 

l«v«ls of si9fiificonc«. In the application of a statistical test of the 
evidence, we are interested in determiniMr, if the observed deviation be- 
tween two groups is too large to be accounted for by chance. The statis- 
tical model that we select will enable us to calculate the probability that 
the results we observe might be attributed to chance. The statistical tables 
which accompany the model will allow us tu read din ctly the ^significance** 
of the evidence produced by an actual experiment. If the probability is 
small that chance variation might have produced the results we observe 
(say 5 out of 100 repeated times) then we an reject the null hypothesis 
(Ho) and accept the research or alternative^ hypothesis (//i). On the 
other hand, if the probability is too large that cliance might have produced 
our observed results (say 25 out of 100 repeated times) then we fail to 
reject the null hypothesis and, of course, we cannot accept the research 
hypothesis. 

Customarily, the research worker states ahead of time at what level 
of significance he will reject his null hypothesis— the amoni^t of risk he is 
willing to take. If he decides iti advance that all decisions will be made 
at the 1 per cent level, he merely disicgaids other results, such as a differ- 
ence being significant at tlie 5 or 10 per cent level, li should be noted 
that there is nothing sacred, other than custom, alxiut the 1 and 5 per cent 
levels. There may be situations in which an individual is willing to operate 
at the 10 per cent level or more. Note, however, that when we reject the 
null hypothesis at the 1 per cent level, we have 1 chance in 100 of being 
wrong in that decision; at the 5 per cent level, we have 5 chances in 100; 
and at the 10 per cent level, we have 10 chances in 100 of being wrong 
in the decision. 

Ptc to ton tllMry. The choosing of tlie level at which we will reject 
the null hypotliesis is known as def*ision theory. There is a rather exl sive, 
and sometimes puzzling, body of ^'t^Tature dealing with the ^autions 
advisable when deciding that a nui. thesis is false (jn^i . >• quently 
rejecting it) or deciding that it is tnu .i: ^ailing to reji Rut the 

operational qualities of decision nuikit. ' e related clo:»w s the con- 
sequences of accepting and acting up. .. e knowledge d( rived from 
research projects, and this is not obscure, nor is it mathematical or 
puzzling. 

In this context, deciding is contingent upon tlie nature of the risks, 
or values, which are involved. For example, much interest has been evi- 
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dented in regard to "curative** drugs for cancer. It is quite understandable 
that people who might prescribe and use such drugs insist upon experi- 
mental evidence secured at a very high level of significance, such as .001 
( I chance in 1,000 of being wrong), or at an even more extreme level. The 
consequences involved in the general use of such a medicine, tested under 
condiUoia where assumed curative properties might too ersily he due to 
chance» are that other treatments tend to be stopped, and the results 
may be disastrous. On the other hand, an experiment concerned with 
temporary or minor changes in diet can safely afford to be based on a 
decision to use a much larger significance levels-say the 5 or 10 per cent 
level. The chances are reasonably good that something useful will be 
learned and, at the very least, serious and lasting injury will not be the 
result. 

In other words, "truth** is a relative matter. The evidence at our 
disposal is always partial, incomplete, and to some extent, tentative. This 
is part of the reason for saying timt "proof" of a positive hypothesis is a 
logical impossibility; there is always some room for doubt, no matter how 
small. Also, what is "good enough*' for one purpose may not at all be 
sufficient for another. And different persons who intend to use the results 
of certain research projects may be willing to a pt or not accept the 
findings as "good enough*' at differing levels of significance. An investi- 
gator cannot foresee all of the situations and circumstances in which his 
findings might be relevant. Since this is true, he may be exceeding his 
responsibility and right to pre-judge the nuitter fur other people by setting 
a decision point in advance of experimentation— a decision point on one 
side of which the results will be declared "significant** and on the other 
side, "not significant.** Of course, the experimenter is free to use his find- 
ings as he sees fit, but he would show more consideration and courtesy if 
he would simply indicate his findings as "significant at the % level/* 
and leave decisions regarding the use of his findings to those interested in 
reading his report. 

Woyt fo iwwdl^ th« hmN hyy u tti t i s . In stating and evaluating the 
null hypothesis, there is one more area of information that is essential to 
statistical testing and to reading the tables of significance whicn have 
been prepared for the various statistical models. This area of information 
is con-^rned with two different ways to state and handle the null hypothesis, 
depending upon the researcher s purpose and state of knowledge on the 
problem being investigated. The two ways for handling the null hypothesis 
may be illustrated through reference to two broad sets of conditions sur- 
rounding all research investigations. 
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The first condition involves tliose instances wlioroin we are able to 
predict wliat kind of differences we will find lx»tween experimental and 
control groups— for instance, that experimental scores will be higher or 
lower than control scores. The research (alternative) hypothesis makes 
the prediction, say, that experimental group scores will l>e higher. The 
appropriate null hypothesis would be that experimental group scores will 
be equal to or less than tliose of the control group. 

The second condition involves those instances in which we are 
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nuT'ly trying to fiiul out il is a iliirt'n'iir*' iM twmi tlir two groups. 

Perhaps wo ilo not know t iioiigli to prcilk t tlu» dircrtion of tlu* iliffcrnm*. 
or |H*rluips oMi mtcTcsts will In* satisficil if wc* siinplv find that thv two 
groups diffcT in any rcspcrt. This tinu» our rese arch hypoth(*sis woiiKI Ih' 
that thf fxprrininital gronp si'on*s will differ from tlu* control groop 
scores (citluT higher or lower). The appropriate* null hypothesis wonld 
he that of no difference. 

In those instance's when*, on the hasis of tlu'orv. pr(*vions experience, 
or other cuvs, \kc nrv ahle to predict the dirrcticni of diffc'rence, make 
what is called a onc-tallvd test, Iv those instances where w<' are not ahle 
or do not wish to pr<»dict .nore than that tluTc will he a difference, we 
make what is called a two-tailed test. These two terms ri»f<T to the tails 
of the distrihntion, as shown in Fignrcs 6 and 7. 

Snp|M)se that we set up an ex|X»riment using a new readirg technique. 
We select two groups: the control group which is taught hy one of the 
conventi(mal cading methods, and the e\|>erinuMital group which is 
taught h\' the new method. Wc have reason to feci, o i the Ixisis of 
newly develo|x»d theory and prior re irch findings, that the exi>erimental 
group will score higher on a |X>st-test in reading which is given follcwing 
a |HTiod of instruction. Tho null hy|X)thesis tluit we t(*st is that the 
exixTimental gronp will have scores <»(pial to or less than the control group. 
But sup|M)se wc are working in an instance* in which there is no relevant 
tliwrv and in which there are no re|X)rts of prior research studies addressed 
to this particular problem. Yet we feel that the new method and the con- 
ventional methml will not yield identical resides. Our interest Ix'comes 
one of testing for differences between two methods^in any direction. This 
time the null hypothesis is that there will Ik? no difference between 
experimental and control group scores. 

When we use a test statistic to discredit a null hypothesis, we are 
using a sampling distribution. This total distribution includes all possible 
values that a ttst statistic can take under the null hypothesis and is repre- 
sented by the curve of that distribution. We are mterested in determining 
whether the sample we actually observe v/ill yield a value which is among 
the values associated wit^ x null hypothesis, when it is true. If our ob 
served sample value is located in a very small area of the entire space 
included imder the curve of the sampling distribution, then we have 
reason for rejection. This small area is know s the area of rejection. 

For a one-tailed test at the 5 per cent level, we have all 5 per cent 
of the area under the curve in one tail. In using a two-tailed test, we use 
both ends of the curve. At the 5 per cent level, we have 2.5 per cent of 
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tlie area in each of tlie two tails, for a total of 5 pvr cent. 

Tlie tables of significance are worked out for Ixith one-tailed and 
two-tailed tests. From them, it is possible to read directly the values re- 
quired for significance at several levels (.01, .()2, .05 10, .20, etc.) These 
levels correspond to the various areas of rejection under the curve of the 
sampling distribution. When the investigator has decided upon his 
research design and U|K>n his hypotheses to test, he is able rather ({uickly 
to determine the necessary significance vahies for either one-tailed or two- 
tailed tests of the null by|X)thesis. 



V'^^Hf Nniiparaiiu'li ii irsis \r\nrsr}\{ Jiin l a|)|)lii .ilicuis nl |)n)l)al 
H^^^l tluMiry to ioin|)ntin«4 si<;iii(jraiK < Arls tor n jri tinn ot Uir nnil 
^^H^r li>lX)tlirNiN. l lir prohahilily Irvrls air nsnallv r\arl rriianlK ss 
^ Wf of tlir sliapr ol tlir |)0|)nlaUoii JislriliMlioiis. But llirsr ron- 
... inalions arr oiilv |)aH i Uir sloiv. lor \\r still arr laml willi 
tilt prohlciu ol M'lri lin«; an appropriair ; o|)UiMiini ^ slalisliral Irsl lor tiala 
analysis. In tlnM)sinti a statistiial U si. wr most ronsitirr Uir nalnrr ol Uir 
po|)ulation Ironi uliiili llir sanipir was Jrawn. Um m tniirr in wliicli (lir 
sc-oH's wrvi' olitainrd. Hir kiml ol inrasnrnnnit alinii^ wlikli was 
rniplovt'il to <'\pr<'ss tlu Morrs. and tlir likr. 

VUv |)rol>|rni of srlrttin<4 an a|)|)ro|)riatr statistiral trst niav hr rrlatnl 
l<) \ari<)ns n siunh ntndithtns wliirli stipulate tlir o|)rialio!ial Jrfinllions 
of llir \aiial>Ks invoUrd and tlir (pirslioiis wliirli ran lir askrj of llir 
data. Ko. tliis pnMiration. rlrvrn slalisliral trsls lia\r hrm srlrrlrd and 
poiipril ill tt-nns nf six ronnnonlv rmonnl* ' wiulilions wliii li nrrrssi- 
tatt' a i lu)iiM' iM'lwt'rn trsls. AIiImmi^Ii llirs* lulitioiis. ami tlir rlrvrii 

srh rtiMl statistiral trsts. lall sliorl of r\liai. >uh- tlir avaiKihIr siippls ol 
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Figure 8. A Minimum of SloHstical Tests To Suil Ordinory Conditions 
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iionpanniirlric- trsts i Siri^rl s IhhjU prcsriits t\\ t'ut\ srs ni ) , tlicy iiicliulc 
sufHc-init IrrwiiN t<> jhTinit analysis of most instancies iMiconntrrrd in 
adniiiiistration and tracliinij. Kignrr S ulrntifi( s {\\v six n scarcli conditions 
wliicli call for diifcrrnt trratmrnts. and tlic various statistical tests wliicli 
arc ap])ropriatc in cacli case. WV will next turn to a description of what 
is involved in nsinj; caili of tlic eleven tests. 

THE ONE SAMPLE CONDITION 

The Binomial Test is nscfnl when a single {population is conceived 
as consisting of tuily two classes. Mere we arc dealing with a "head or 
tails" situation where, for example, the prohahility of a wius landing 
heads op is 1 2 and the probability of its landing tails up is also 1 2, so 
the pro|X)rtion (P) to In expected for obtaining either a head or a tad 
when tossing a single coin is also I 2. But what is to Ik* exi>ectcd if we 
toss 20 ct)ins sininltaneonsly? If the null hy|X)thesis is true, we would 
expi^'t to find alxnit half of the ct)ius landing heads up and half of them 
landing tails up. when tossed a fairly large numlH*r of tinu^s. 

But when we do research, our statistical interest is in determining 
the prohabilitv ol htaining the obseixed valnes—say 6 heads and 14 tails, 
instead of H) heads and 10 tails—in a toss of 20 coins. This probability 
can 1k' computed hv applying the formula for tlie binomial disfrihution 
which is (p + where /> represents the pro))ortion (frequency) of 
heads and (f the frc(|nency of tails, and n is the number of cases in tlic 
sample (l)oth heads and tails). The binomial expansion can Ih* c*omputed 
to determine how often a sample as unusual as the one observed wotdd 
come from some hypothesized |x>pulation wherein we are testing a de- 
lilx*ratelv introduced indepcMident variable (say psychic |X)wers of tlie 
coin-tosser). 

However, binomial probabilities for various combinations of two-class 
events have conveniently lx*en tabled, and from one of the tables we can 
read directly the probability of most actual occurrences. Use of the table 
obviates the necessity for computing the bir uuial expansion. The tables 
generally let the sym1>ol x ecpial the smaller numl>er of observed frecjuen- 
cies, and the symlK)l .V e<pial the total number of frecpiencies in the 
sample. For example, referring back to our mention of obtaining 6 lieads 
and 14 tails from a toss of 20 coins, what is the probability of obtaining 
this exact distribution? Here 6 and N = 20. The tabled probability 
for these values is .058, or significance is reached at slightly more than the 
5 per cent level. To continue wiHi these examples of two categories from 
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the same population, other tahled vahies which we might ha\ o (Kcasion 
to use are: N = 8,x 1. f (probabihtv) .0.^5; N 21, x P xm, 
.V = 48, X = 17, P - .030; .V = 100. x 41, P — .044. As we can see 
from these illustrations, some binomial tables allow ns to work with sample 
sizes between 3 and 100. The tabled significance vahies are one-taih>d. 
When a two-tailed test is desired, tin" significance \ ahie is doubled. Thus, 
when iV = 20 and x^6, the two-tailed probability is 2(.058) = .116. 

There are natural populations in the school world, the mcasnrenients 
of which may be dichotomized and analv/ed through use of the binoniial 
test. Examples of such categories are: boys and girls. Negro and white, 
stars and isolates, teachers and administrators. For snch easels, all pos- 
sible obserx'ations from the population will fall into either one or the 
other of the two classifications. When the proportion (frecjuency) of cases 
in each of the two categories ec|uals 1/2, the prepared tables may l>e 
used. When the frequencies in the two categories are not ecjnal to each 
other, they may be expressed as proportions and substituted in the formula 
for the sampling distribution of the binomial. But to promote simplicity 
and parsimony in computational labors, it is generally preferable to select 
samples and designs which give an equal number of randomly selected 
cases for each of the two classifications. 

Ihm Om ScNiipW Chi Squora TmI is suitable when the res(*archer is 
interested in the number of subjects in a single sample which fall into 
'^rious categories or classes. The number of categories may be two or 
more. Chi square is a test of the significance of differences— in this case, 
<lifferences between categories. 

As an illnsrration, suppose that a congenital right-liander tosses a 
coin 100 times with his left hand and keeps track of the results He ob- 
serves that 36 heads and 64 tails appear. We refer to these frequencies as 
the observed frequencies. Next we state the null hypothesis that this 
distribution does not differ from what we would expect by chance, or 
50 heads and 50 tails. Thes' ♦ uMicies are called the expecled (or 
theoretical) frequencies. Tlx m A and the expected frequencies can 
be compared through recording them next to each other in what is called 
a contingency table, like this: 

T«bfo 1. CoMpprisofi of Obt«fy«d cmd Exp«ctttd Fr«quMciM in 100 Coin Tottot 



We tv%i miT null liV|)i>tlM*<«is hy luing flu* goruTal formula for clii 
M|iiarc: 

. . ^ (olpMerted frequence vxwried ffeiiuvnvtif 

cm 9u%mfr the of. . . 

^ exftecfeft frcquennj 

TIk? formnlu r(H|nirt*» flint wo takr r«cli oImmt%(h1 frtH|iwncv, »nl>- 
triic*! from it IIk* i'<»rr€»»|M»ii(liii^ rx|Mt1ftl frr<|iu*iK'\ , Mpwiro thv illffrrnKT, 
iind di%'id€* iIm* rvsuh l>\ tlu* rKiMiitil fr«*<|nnK*\ . Tlu* tnni of tlu*u* optTU- 
tiont it clii »i|imrr: or Tft 50 14^ 196 W .1.92, ami 64 S« - 
14' 196 S(> 3.92. aiHl 192 \ 3.92 7.H4. wlikh i% tlie olHaimnt chi 
M|iiarr in thin cauv \Vr coiniMirt* this compotcHl %aliM* with the %aliiei 
shown in a table of tk* DifttritMitiiMi of Chi S(|iiarr. TIk* tabled compari- 
aon ahowt tliat oiir «>imT%atkMiik are kiKiiiflcant at tM*tt«T than tlie i per 
cttit level which (><|iiaU a chi Mfuare %aliie of 6.635 Otir ctmiptited value 
is larger tlian this, Imt MiiallcT tlian tU* IO.H27 re<|iiireil f^ir significance 
at the .ml level. 

In this ilhistrati<NK we clt*tennined t\w lev(*l of significance by means 
of tlie geiMTal formula for chi s(|iiarf*. We could lia%'e accomplislied sub- 
stantially tiM* same thing thriMigli a|>plicatian of tlie iHtiomial test. Using 
tlie same fre«|iietK*i<*s in oiir c<Hiting(*nry talik*, we ikh' that we liave an 
S (total fre«|iH*ncies ) of HN), aiid an x (tlie smalk*r (resiliency) of 36. 
Referring to a tahk* of signiftcamr vahM^ for tin* liitMHoial distribution, we 
Ind that wlien S KM) and x 16, P - .(103. 

At this point, we liave to hv cuiK^emed with tin* tails of the two dis- 
tributions if we arc* to compart* tlM*m* The chi scpiare table is computed 
for two- tailed tests of tlie null hypothesis, and tlie binomial table is com- 
mited for one-tailed tests. As we liave already seen, we can convert the 
Mmmial vahies to tlieir two-tailed equivalents by doubling the significance 
values shown in the table. Thus, the tabled value of .003 becomes 
2( 003)^ .006. Our obtained chi s(|iiar« significance level was higher 
than tlie .01 level and something less than the .001 le\'e|. Tliis compares 
quite favorably with tlie two-tailed signifkancr k*vel of tlie binomial test. 
In both instances, we can be quite confident that tliese results are different 
from thoae produced by chance alone. 



IW SIpi T6tl is the simplest of all the nonparametric tests. Calcti- 
lalioii of the probability of an observed distribution requires nothing more 
conqilicated than the simple counting of positive and negative signs, and 
then referring these vahies to the appropriate table of probabilities. But 
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fit M'tirt h for tfu rntrtttutfu r in Utiut aiion 

nn n olv^l and Vl..gaut." Tlu- cka, apinal <,f this .n,d;,th.n..l trst is 

;;::;':;:|> " a-ui pil;:L;; 

In many t>|H-.s „f iuvrstiKat ,,nantitatix ra.s„rr„,nit.s ca ,t 

K. ma, ... .., ..v. n a c I. tr ranking ..f tlu- n.atrrial n.av n.,t Ik- .K.ssihl.. 

i. cr;..r'l ' *'Mx rin,n.tal and c<n tn.i 

ro T 1 with its 

•..ntn,!. Tho .mlv r..,,„,rn.u.nt is f.,r . ach pair t«. Ik- n.atcl.il with r.-siH-c t 
M X. ago. ig. „r otht r r. lrvant charact. ristks pertinent t<, thr s ndv 
Ik ng cnchK... As w. hav.- .Km.l In-f..... on.- wav .,f acc.n.pli g 
tins ..s o „M. ..ac , M.hj.vt as his „wn ct.ntml. If „|>M.rvati«ns „f oU ,uit 
ca . n.|iahlv „Kl.c.at.. ov.n .H„„i„al difrorc-ucs snc h as •|H.tt..r-w J" ' 
f..strr.sl„w..r. gr..at. r-f..w..r.- etc .. th.- sign t.vst is . ticniarlv uM ' 
CM c«nrsf tlK-r.. may Ik- instancvs in which wr do not haxr „nantita- 

H.m s for th.. two nM-niln rs of a n.atch.tl pir, For rxanml... i„ U.th thr 
n<nK,na„t.tat.vj. and t k- ..nantitativ. cjpari.sons. wc „!av 1. 
wit , a rscarch h.v,K.th,.sis which ass^-rts that thr rxp. rim. ntal trratn,. »t 
W.II y...|d r..sn ts .snp,.ri«r to tin- c.ntrol tr..at„u.nt. This, of corse 
or a o„...tad. d t. st. TIh- m..tlHKl consists in finding th.- chff. renc.. Ik- w^n 
an o «..rvatum or sctm- for a nH-n.^-r of the ..xpcn„,..ntal Rn«,p and t ha 

of t H-*^. difr. renfes ar.. positiv.. and lum manv are negativ.-. WIkm, scores 
are tied for any pair, tin- diff..rence i, zero. In tin- event of ti.xl scor.H" 
w-roi a„. dropped from tin- analysis ami the S is reduced. TIm. 'n.,11 
hvpotlH^i,, of co.,r«.. is that w.- wotdd expect to fimi alKM.t half of the 
differenc-e. positive and l»lf of tlnm m-gativ... //.. i, rekx-ted if too few 
differences of one sign occnr. 

To illustrate tlM- applicati.m of tin. sign test, im .gin*, that we are 
working in an eighteen-teaclier .lementarv scIkwI. Oor problem is to 
.mprov.. teaclK-rs- attitudes about facultv meetings. Based upon the in- 
volvement principle, we by ,K>the,ize tlut if tin- teacln^ pUn and cond.Krt 
their own meeting., instead of contiuuing tin- cuitomarx practk^ of at- 
tending tin- prim-ipar. nM^ting." tin y will reflect an increase in «.tisfac- 
tlon with staff meetings. Tlie null by,MHlH^is would be that we co,dd 
expect about as many dissat.sfactkms as satisfactkms with tlie new 

"^TT'Ll" " '""ling', we constn^t a simple 

satisfaction-dissatisfaction scale: ' 




Hclorc tlir « liiiiinc In wc ;iNk •■;u li l«';u li«'r to «'\|)r«'ss his f«'cliiii;s 

(-(iiK'cniiitt; llif niNtoiiuii \ ° priiiciiKil s iiiccliiij;;." Tliis is tloiu- on a prr- 
sitrrt ol paper, aiiii tlic sliccis arc |)r»s«'r> rd to svrw laU-r as 
coiitiol scores. Alter tlic cliaii^c is iiiiitlf. \vc ai^aiii ask cacli tcaclicr to 
rrt-onl !iis Iccliii^s l)\ tliMuiii^ a circlf aroiiiid tlic \ tliat most iicarlx 
reprrs« iit\ liou Iccls. Tlic iiniiirrical w cijilits— ( 5 ) (4) (3) (2) (1) — 
tlo not iM (-(i to a|)prar on tlic scale prcparcti tor tlic tcaclicrs. I)iit tlic use 
of scale v\<-i^lits lielps in rccordini; and interpreting tlic |)ositi><- and 
ii«'j;ati\e signs. 

This pr«K-ediirc i;i\cs iis IH inatclied pairs, and it also giv<'s iis a 
■■|M'fore-after with control uroiip <lesigii. 

NtiK sMp|)osc further that we sininnari/e onr data l)\ filldin^ the 
diflcrciice lH-t\\ecii the first aiitl second scaled reS|M)nse for each teacher, 
taking care l«> sniitract each time in the same order ( sa\ s<-cond res|X)iise 
ininns first res|)onse). In liii<< with onr r<*searcli li\ |M)tli<*sis, |xisiti\'<' and 
n«*^ali\f difftreiices will indicate increascN and di'CTcases in satisfaction. 
When we record air findin^s. assiiine tlwv l«x)k like this: 



T«M« 2. Potitiv*, N«gativ«, and Zero Signi for II Tcochcri 
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By cxiiinting tlies«« signs, wt* olwcrve that 2 teachers sliowed no change 
iH'lween the tww treatments (0). Tliev are drop|X>d from analysis, and 
the .V is rediicM'd to 16. Tlu-re are 4 negative signs, and thes«' are repre- 
senltti liy tli<* .syndM>l x. (Consulting a table of distriliiititin for the sign 
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test, we observe that for *V ^ 16 and x 4, the level of significance is 
.038, or the null liypotliesis can Ix* rejc^cted at slightly more tiian the 

3 |HT c*ent level (one-tailed). 

It should be apparent that what we were doing is analogous to the 
binomial test, and that in effect we were seeking the probability of tossing 

4 tails (or heads) out of 16 tosses of a c^oin. But the sign test [Hermits us 
to deal with a two sample condition, instead of simply a tnie sample casi\ 
and this intrmlucTs us to the classical and most often used circumstance 
in the entire range of research design and statistical analysis. 

THI TWO INDiHNMNT SAMPUS CONDITION 

TIm MtNHi-WMtMy U TmI is one of the most powerful of tlie non- 
parametric tests. "Power" here refers to the relative efficiency of this 
statistic in rejecting the null hyix>thesis when that hy|K)thesis is false. And 
the null hypothesis tested by the Mann-Whitney U Test is that the two 
indei^endently drawn groups are Siimples from the same population. 

This test is used with independently drawn groups, and the size of 
the two groups iK*ed not Ik* the same (although it may be). It is used to 
test the difference betwet*n the distributions of scores lor tlie two groups, 
or the difference between two methods or treatments applied to the 
members of an exix'rimental group and a control group. Scores must be 
expressed at least in ordinal measurement scales to permit ranking. In 
fact, this is one of the most frequently used nonpar;^ metric "rank tests.** 

The procedure is quite forthright and uncomplicated. When measure- 
ment has been achieved for both the experimental and control groups, the 
scores of both groups are ranked together in one composite distribution 
from k>w to high, assigning i\\e rank of 1 to the score which is algebraically 
lowest (if negative scores are present), rank 2 to tlie next lowest, and so 
forth until all scores receive a rank. Tied scores are assigned the average of 
tlie tied ranks. During this joint ranking, it is important to retain the identi- 
fication of each score as £ (experimental group) or C (control group). 

Tl)e next step requires that tlie two groups be separated, each score 
now being assigned the rank determined for it irt tlie composite distribution. 
The smaller of the two groups is identified by the symbol ni, and this 
may be either tlie experimental or tlie control group. The larger of the 
two groups is labeled n^, and of course may be eitlier the £ or C grottp. 

Next, the ranks for each of the two groups are summed. The symbol 
Ri is used for the sum of ranks for the ni (smaller) group, and the symbol 
Ri signifies the sum of ranks for the (larger) group. 
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Wlien till* vaWs of rii, rij, Au ami R:i are known, tlio valiir of V may 
l)e (leterniiiu*d. This is iloiu* tliroiigli making snhstitntions in two formulas. 
The two formulas vielil different values for V. It is tin* smaller of these 
two values that is used to detennim* significance. TIk* symbol l^ is used 
to represent the smaller value, and the symlM)l deiMites tlK^ larger value. 

The metliod for detennining tlie significance of the ohser\ed value 
of U\ depcMuls on the size (nnmlier of cases) in the ?ij group. Tables have 
been prepareil for one-tailed ami two-taileil tests of nj — 3 to n- — 20. 
For groups from 3 to 8, exact probabilities are tabled; for fij groups 
between 9 and 20, critical values of V are tabled; and for n* groups larger 
than 20, an approximate test leased on the normal curve is available. It 
has been shown tliat when iii and nj are larger than 20. the sampling 
distribution of V rapidly approaches the normal distribution. 

TIm Mtdian TmI will provide information on whether two independ- 
ent groups have been drawn from populations with the san;e median. It 
is applied when individuals are assigned to the two groups (and conse- 
quently to two experimental treatments) at ran Jom instead of by matching 
as is the cas<* with the sign test, Tlie null hypothesis states that tlie two 
groups are from populations with tlie same median. The research hypothe- 
sis may be that tlie median of one group is different from that of the 
other ( two-tailed ) or that the median of one grotip is higher than that of 
the other (one-tailed). The number of individuals in each of the two 
independent groups does not necessarily have to be tite same. 

The procedure closely follows the intent of the test. The idea is to 
compare the numbers of individuals in each group whose scores are above 
the median score of both groups ranked together and the number of indi- 
viduals wliose scores fall at or below the composite median. If the two 
groups are samples from popidations with the same median, we would 
expect about half of each group to have scores above the composite me- 
dian and about half to have scores below this median. 

To |>erfonn the median test, we first determine the median for the 
combined scores of both groups. Customarily, this is* accomplished by 
ranking the scores from lowest score to highest score (or vice ^ersa). 
The median is that value which has as many cases below it as there are 
cases above it. There are just as many cases with values below the median 
as there are cases with values above the median. When there is an even 
number of cases, the median is the value halfway between the two central 
numbers. For example, if we had an N of 40, and central numbers of 19 
and 20, the median would be 19.5. To permit ranking, the scores must 
be expressed in at least an ordinal scale. 

II 
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Nrxl wc liMiiit niiiMlH r of sion s ahovr tlir joint iiu dian for imcIi 
of ihc two poiips anil tlir niimhrr of sions which fall at or In low thr 
•nrdian lor i\u U of tin* two uronps Thrsr xahu s arc' nitrrcil in a 2 - 2 
or foiirfolil loiitinmMicx talilc mkIi as thr h)lh)wiMV 



TobU 3. 2 • 2 Contingency Tabk SKowing Placement of Frequencies and Totoli 



If lK)tli group f and group II are siiniplrs from |>opnlatioii.s with the 
sainr inrdiaii, wr wonid rxix-ct frinpinicirs r/ and r to he alxHit equal, and 
free|nencies h and d to Ih» alM>ut (H|ual, Tlie siunifiiana- of discrepancies 
from this exiH-ctation can 1h» determined hy snhstitntiiijr the ohservrd values 
derived from an actual exix-riinent for the letters r/. h, (\ and d in the 
wntingenev table, and llien calculating the value of chi sipiare. A formula 
is available for this purjKwe which lends itself to machine computation 
and which avoids the necessity of calculating exptxted fre(|uencie.s. This 
is not the general formula for clii square which was used in connection 
with the one sample chi Miuare test, and it mav Ix- used oidv in the instance 
of data which can Ik* set in a 2 \ 2 or fourfold contingencv table sncli 
as the one just presented for the median test. 

If a two-tailcd test of the data is desired, we use a table for the Dis- 
tribution of Chi Square to determine the level of significance. This is 
done directly bt^cause the chi square table of significance \alnes is put up 
in terms of a two tailed distribution. If a one-tailed (directional) test is 
desired, we halve the level of significance shown in the table. For ex- 
ample, according to the table of significance, an obtained value of chi 
square of 2.706 or more (and less than 3.841) is significant at the .10 
level for a two-tailed test. Halving this, we have .10 2 = .05 for the signifi- 
cance level of a one-tailed test. It is important to note that a one-tailed 
chi square test is permissible only in the instance when a 2X2 or fourfold 
contingency table has Ifeen employed as a way to set up the data. All clii 
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square* tests wliicli deal witli more than two groups or more than two 
clasisificatjons nmst follow the general proeednres for two-tailed hvjxitheses 
and decisions. It is not i>ossil)le to convert chi scpiare contingencv taMes 
larger than 2-2 to one-tailed tests. 

We will next turn to the chi scpiare two inde|H»ndeiit samples test 
and take a brief look at 1) determining expected frecjuencies; 2) further 
nse of the general formula for calculating the value of chi S(|uare; and 
3) using the significance table for the Distribution of Chi S(piare. 

Th« Chi Squor« TmI for Two Indopondont Somplot ma\ used to 
determine the significance of differences lK?tween two inde|H*ndent groups. 
This test may be used when the data can 1k' expressed as frecpiencies 
(number of cases) in separate classifications (discrete subgroups of the 
total sample), and all cases can be accounted for among the classifications. 
The measurement involved may l)e as simple as nominal scahng, or ordinal 
and interval scaled data may Ix? used if desired. Numl>ers of cases in the 
two groups and in the several classifications do not nwessarily have to be 
the sanie. 

The research hypothesis to Ik? tested usually is that the two groups 
differ in respect to some characteristic, and therefore in respect to the 
number of cases which fall into the separate classifications. To test this 
hypothesis, we count the number of cases from each group which fall 
into the various classifications, and for each classification, compare the 
proportion of cases in one group with the proportion of cases in the other 
group. To do this, we use the general formula for chi scjuare exactlv as 
it was presented for the one sample chi square test. 

But now we are faced with two procedures, in addition to the use 
of the general formula, which were not pointed out in the description of 
the one sample case. The first procedure concerns a method for deter- 
mining expected frequencies, and the second procedure concerns the 
calculation of what is called "degrees of freedom" which is necessarv to 
the use of the Distribution of Cbi Square table of significance values. 

To illustrate these procedures, a simple example of a research problem 
involving chi square will be given. We will be using the general chi 
square formula which directs that we take each observed frequency, sub- 
tract from it the corresponding expected frequency, square the difference, 
and dividr the result by the expected frequency. The sum of all these 
resulting quotients is chi square. 

Suppose that we wished to test differences between the 43 sixth grade 
girls and 52 sixth grade boys (N — 95) of ouq elementary school in 
regard to their preferences for science, dramatics, and band. This school 
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maintains a schedule of dubs wliicli tlu- childri-n max i-lect for three 
meetings per week nnder faculty guidi-nee. We w isli to test whether girls 
and l)<)ys differ with respeet to their interests in these three cinhs. as 
expressed hy their elective affiliation with one of tire-ni. We set np a 
fre(piencv connt of the ninnlx-r of girls selecting each dnh and the nvwl 
l»er of lK)ys selecting each cinh: 



Tabic 4. Girli" qnd Boy«' Choices of Interest Clubs 




The nnll hyiwthesis would Ix- that sex is independent of interest i e 
that the proiKJrtion of girls selecting each duh is the same as the pro- 
|K>rt,on of lK)ys selecting each chil) when the total niemlH-rship of all three 
cluhs IS c(.nsidered. But it is nec-essarx to detern.ine the expected freouencv 
for each c-<iunt tabulated aliove. 

To do this, for each ohserxed frecpicncx lec-orded alxjxe. we multiplx 
the two totals in the n.argins common to a partictdar frecn.encx . and then 
clix ide this product by the grand total, \'. to obtain the expH^ted frecniencx . 
For example, the observed frequency of girls selecting the science cliib' 
IS 12. The total in tbe rigbt-l.and margin wl.id. is common to 12 is 44. 
the total m the Iwttoni margin whidi is common to 12 is 43 and N = 95 
So we have 44 X 43 - 1892. and 1892 95 - 1P.9. Hence, wc have an ob- 
served frecpiency of 12 and an exjiected frecpiencv of 19.9. In the case of 
the number of Iwxs who selected the band cliib. we nudtiply 15 X 52 (tbe 
two totals in the margins common to 6) and divide tins product by 95, or 
780 95 = 8.2. We continue to do this same multiplication and division until 
we have determined an expected frecpiencv for everv observed frequency. 
At tbis i>oint, it may be pc-rtinent to interrupt with the observation that 
the expected frecpieiK-ies amount to a numerical statement of the null 
hypothesis: these are the frequencies we would have had if there had hecn 
no differences. 

VVhen we have the observed and tbe expected frequencies, we can 
complete the dn square calculation which directs us to subtract each 
expected frequency (E) from its corresponding observed frequency (O) 



StattsiU'ot Trsis 



s(|nan' llic iliff«T< iK<'. ami dividv In ihr v\pvi U i\ (nM|iicm \ . Tin* rrsiths 
rt tills «)|MTaU()n for all pairs of ohs<T\('(| and i xim'iUhI frc(|ncnrics arc 
snniincd. and this is clii sfjiiarc. 

Wlun snhlrailinn ihv iApriUil from ihr ol)s<T\<d fr<Hmcmi< s. sonu'- 
limes ihv diff«Triir«' is a lU'jjalixi' lUunlHT. Bnl llus rans< s no diffirnUy 
lM*raiisc llu' \\v\i sir\i is lo s«|nan' ihr diffi'irntc. and n<'^ali\<' iininlxTs 
convcMiinilK disap|MMr. Tin* sipian* of any nnnilx r. posilixx* or nciralixr. 
is always positive. 

Thr coniplrti' svi of sUps in ihr coiupntation of ilii stpiarr for oni 
problem concerninjj si\lh jjraih' eliihs follows: 



Tobl« 5. CatcMlatian af Chi Square far Data af Table 4 





To drtcTmin<» tlu' K'vel of sipnficant«» for any ohtainc'd t'lii s<jnare 
\alnr. it is nec<\ssar\ to rompnt<» ihr decrees of frc'edoni ass<Kiateil with 
that value. (Ihi stpiarc* is really a faniilx of distrilintions; that is. thcr*' is 
a different sampling distribution for rvrry \alne of decrees of fr<»cdoni. 
The sijinifieanec- tahh- for the Distrihntion of Chi Scjnare lists the various 
nnnilxTS of dejjn»es of fre<»dom (nsnallx from 1 to xertically doxvn 
the left-hand tt)lnmn of the tal)le. When xve knoxv the dejjrees of free- 
dom ((If), ysv read horizontally into the table opi^)site the appropriate 
listinvj where xve find a roxv of ehi s<piare xalnes. If an obserxcd xahie of 
ehi s(|inire is eipial to or greater tlian the \alne jjixen in the table for a 
partienlar lexd of significanec\ at a partienlar df, then //.» may be rejeete<l 
at that le\el of significance. 
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How is the nnmlKr of dcj^recs of frcidoin calculated? In anv eon- 
tinji<^ncy table eoniiK)sed of R rows (classifications) and C columns (groups), 
the nninlKT of degrees of freedom is gi\en hy (R - I) (C - 1), in our 
example, we lia\e three rows (science, dramatics, and hand) and two 
t^olnmns (girls and l>oys). Tims R - 3 and C ^ 2, The number of degrees 
of freedom is (3 1 ) (2 1 ), or 2 ^ 1 ^ 2. We are working at 2 df and 
this enables ns to refer to the table for the Distribution of Chi Square to 
determine the level of significance. In our example, the obtained value of 
ehi M|uare was 10.67, At 2 df this vidne is larger thun the tabled value 
of 9.210 which is significant at the .01 level, but less than the tabled value 
of 13.815 required for significance at the .(K)l level. Hence we mav decide 
to reject the null hyixithesis at slightly In'tter than the .01 level. We may 
ctjnclude that there is a tlifference in girls* interest and Ijovs' interests as 
expressed by our data. 

For a contingencx* table ct)n«iM)S(*d of two rows and two columns, 
referred to as a 2 < 2 or fourfold table, the nnndxT of degrees of freedom 
is (2 1 ) ( 2 1 ), or 1 y 1 1 The 2 < 2 table is a spt^cial case always 
having r// = 1, You will recall that this set-np is essential to the median 
test, and that it represents the :iingle instance wherein chi square can be 
con\erted to a one-tailed (directional) test. 

THE MORE THAN TWO RELATED SAMPLES CONDITION 

Thm FrMitMin Two-Woy Anolytit of Vorionco by Ronks is useful 
for testing the significance of differences among three or more related 
groups. The null hyi>othesis to be tested is that three or more samples 
have In^en drawn from the same (Population or populations with the same 
median. The restriction is that the several groups must each consist of 
matched indixidnals, and this of course results in the same number of 
indixiduals or groups in each sample. The matching may be achieved hy 
the researchers selecting three or more groups of subjects matched on 
certain relevant variables (age, !Q, sex. socio-ix*onomic status, etc.) and 
then randomly assigning each individual or group to one of several experi- 
mental situations. 

The data are arranged in a two-way table containing rows (which 
corresixmd to the individuals or groups selected for study) and columns 
(which eorresix>nd to the expi^riniental situations). Thus, we work with 
a table of \ rows and C columns. For example, if one wished to study 
the differences in learning achieNed under four different teaching methods, 
\ sets of four matelu d students might 1k» selected, and one student from 



ratli srt \\niilvl l)r assi<4m il to iiH tlnul I, aiiotlirr fn>ni tacli set to nu'thod 
II, aiiotluT from racli st't to iiu^tliod 111, aiul tlio fourth to inctlioci >\*. 
Siipposr that \v«» wish to stiuly tlw stores of six groups iiikUt the tour 
tfadiiiij; nu tlMuls or situations. Kach group contains four uiatdiccl stu- 
ilfnts. OIK" iM'ing assignt'cl at random to each of the four situations. Further 
suppose that thf st-ort's for this study an* those given in TaWe 6. 



Table 6. Scores of Six Matched Groups Taught by Four Methods 



M II t « 
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To aualv/e tlu's<» «lata using iliv t\vo-\va\ analysis of variance by ranks, 
\vr rank the ohst rvations in «Meh row from 1 to 4. giving the lowest score 
in each row tin rank of I. th«' next lowest the rank of 2» and so forth. 
For example, thr four ol)s<T\ati<)ns in the top row of Table 6 are 22, 14, 6, 
and 19. These replaced by ranks 4. 2. 1, and 3. The ranks in each 
column (situations) are then suunned, thus obtaining the data in Table 7. 



Toble 7. Ronks of Six Groups Taught by Four Methods 
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If the samples (c^)lnnins) have Ikhmi drawn from the same iK)pnlation. 
the ranks in eaeh c^himii will Ik* a random arrangement of the nnml>ers 1. 
2, 3, and 4. Under these circumstances, the sum of ranks for cohmins will 
tend to l>e the same, or nearly so. But if the snhjeets' sa)res have hcvu 
dependent on the various teaching methods (situations), then the rank 
totals will vary from one column to another. If the totals differ signifi- 
cantly, then //o may he rejected. 

The forniuhi to Ik* applied to the column sums of ranks is Friedman*s 
chi S(juarc test. The values given hy the Friedman formula are distributed 
approximately as chi square with df = C (columns) — 1. In our example 
we were working with 4 ct)lumns, or df ^ 4 — 1 = 3. 

The general table for the Distribution of Chi Square can lie used when 
the numtxT of rows (\) and columns (C) is not too small. For very small 
.V and C, a table of exaci probabilities is available for C = 3. *V = 2 to 9; 
and for C = 4, .V 2 to 4. 

When the investigator is working with individuals not organized into 
groups, it is feasible to substitute individual subjects for groups in record- 
ing row scores. In such an instance, the same sample of *V individuals is 
tested under each one of a number of different ex|x>rimenta! conditions, 
and matching is achieved. Here N ecpials the total numlier of subjects 
(rows) and C ecpials the number of different experimental conditions 
(columns). The procedure is the same as that just descril>ed for the 
case of groups randomly assigned to various situations. 

Because the Friedman test, b> definition, is always concerned with 
more than one degree of freedom (three or more groups), tlie sampling 
distribution of chi scpiare always is analyzed by a two-tailed test. Hence, 
the research hypothesis is capable of predicting an over-all difference, but 
not the direction of that difference. 

THE MOtE THAN TWO INDEPENDENT SAMKES CONDITION 

When three or more groups or situations are to be compared in an 
experiment, it is helpful to use a statistical test which will indicate whether 
there is an over-all difference among the groups or situations being studied. 
If an over-all difference of sufficient magnitude is found to permit rejec- 
tion of then any pair of samples may be picked out to test the signifi- 
cance of the difference between them. Otherwise, the two-tailed nature 
of testing three or more samples approaches scientific value chiefly as a 
hyix)thesis generator for more explicit (one-tailed) tests and decisions. 
We have just considered procedures for testing three or more random sam- 
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wmy iIm* %iiliM'% <i4 iIm* frfiM-njIiiMM. W'v Umll iitm imniiilrr two 

ptvAtnliifv-ft Urn UMmg llinv tm mmmr ftniiiplit wkk-li «fr MUf iwIcI m J mhI 

«ir|ik-li nn' MJi tif iIm* %iumt %kir 

iM^ul whrM wr wii4i lii lr«l tli9rr <ir Min* kmdr^'mh mil %iMiiplr%. iiiil til 
tkr MMnr mA mil m<li4irtl tm tlir K«%K til 4 pruiri \«ri«Mm Tlir 
KniiLil*W«llb lrt'lift^|<M* tr%t% tin* ihiN li\|Mi||ir%i«. 4% tliH-% thr ^rk tl mn 
lrr(iiik|iir, IImI lltrrr %m imifr wnmi li*% imlnmm Afr iWawii ffiim 

thr iMir pnpiilttlliiii tir ffiim |iii|iNUliiim milk U m i U r «\rf«|(r% 

To appK tkk Irtt. sN nf tfir tc^itrt Uw tin* tkrrr «w miirr Mmyh i avr 
rinlMid In' ranks tn « «iii|[|r irrlrt f Tanfr J Inmk km tt^irr lii li^ tctitr 
(if rounr. «t Irwl i i wIiiMil m mm t^m mk iil m|iiifnl lit prrmM tlir ninkiiift 
ol imm. Tkr kiwrvt %aliir It mmkpm^d m nmk tif I. thr rant lim<-M « rank 
of 1, and to on. Thr wmrn of nmkft. Ai. Ai. ||« . . . A* hw rarh til thr C* 
■■w yh I, k iiiilaini d. Ulim tir« ormr. thr mmoI romrMHin ii odofilrd 
of Milydiiii lo Ihr Mrd cdMrr% olioM ihr o%vfMr <if thr rMki thr% mrndd 
hovr occtni4td hod thrtr hrm no Mrt. \ rifiah tkr lolol o u iwlMr of com 
in ■anf li 11 mwditoid. 

A fliatlflllri fit It cvlmhiird ftom Ihr data- l^o* forwida wtrd rrttdit 
In a vahir whlc^ It dhrttfbnlrd aa c4il ti|norr wMh d|f C* I > Wi/r IPt^ 
V thr DMfffknIlon of Chi Si|narr con hr ntrd to di^rnnlnr 
povM UmI Hit ttM MfimC mr fm mmM. 

3 Md IW Ill r of CMN to mHi of tW |Im«v MMpIn It 9 

Mm i^ivv, inr cm H|noi^ a|^mMmionDn it noi wmKK^mmmj ciaar. nfninai 
ano vVMnt navr pio^nini a iflmr cn noi^ nvonnvoiivm nn vvfwt noHmtr 
vahiaa of Ot* nt* and Ot orhri^ tiart of Hir thrrr taniplra contM of ol 
Mom of S. 4. 1 m4 I 
of IraHnii Ihrrr upanpt lof diflrrmrrt In a 
hr cfiird ftani a nnl%vrtlt% conftr In 
Hon. ChMM m 

In X-Y*Z JUtlts 

on thr tnh^rrt, and ptlof In mlrtlnf Into trndnof typo ditraaiiontt a 

li) iiyiiliHiii<iiili. aMi 
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IIm* It'll «itMiU tmlk'alr trtilKr Un*ti||tli iif iifn^*tiM*iil 

Titlilr H prr«ml% iIm* %(tiri*% iliHi*fifiiiit^ Inmi llit* Ht%lriiMM*iit% 
Ttir 21 Mtin*% wrtr tlitti rutiluxl lii|{i-llii*f fnnit l«i liiflM'%1 In iiIiIjMi 

llir MNlift UnMiN In T^lilr V 

FnuM llif%i* tUU. Ilir MiilMli' II ttitii|Militl In MilMlliillittf llw 
€4iM<f%r«l \«kii« Iff ikr kniftluit\%'«illH UmnmLk TIm* ttiiii|itil4|iiiii h-miNmI 
III II 1174 KnTMHT iIm* iHimlit*f iif MMll\itliMK Ml I'ji li «il IIm* tUn^' 
l|riiii|i% iftAt lurfrf IImmi \ ikr irfilAiiM'il \iikii* iil II 4I il^ I I 2 
n^lrtml cUmlH l«i m lulib* l<ir IIm* llMrlliiHiiiii «il IIm Si|iMrr Tlii« Ijlik- 
Unmnil iImI mMli Imci 4k^i*% «i( liM^ikmi. 411 II ^4kM ct|«Ml lo m Kri*j|«i 
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lliaii HiMilfl In' iHf*<lril lo rr.K li %iuiiilk .iik r ;il llic .0 lr\rl I lu 

<il»(uhM^I %.iliir of // in lliis rvini|»k> vi.ih k'^n lli.iii llii^ • VI7l \ Hit;iiili 
Vik^Ht* rruc lMHl .il .1 IrtI lliaii |In> . TIn* <l<*i'i\Mni in lliis 

«4M* Ma% iImI //.. (<Nil<| iwil In* rr|«H'lt*(l. TIn* llim* ^nNi|H (»f si>lNN»l 
•MliiiiiiKlralor^ ilUl iw>( iliHrr %it(iiifM*aiilU hi m;*inl to ulliliulrs on 
\-V-/ aliilil^ ^r<Ni|>iii^. 

1km IsMMteM mt Htm Mtorftan TmI is .1 ^impk r HNilNNl for Irslini; 
|Im> ihffrrf*iK*«'% lN'lHi*«>n lliri*«* tn \wwv iiNk*|M*iKl<*iil vnn|ik's. Tlir <l.il.i .irr 
(*«Nii|HiM^I III C \«iiii|ik*\ (»f M». n.. . . iiI>mt%4Immi\. As iN^lorr in 
rrf(airil lo llircv iht nmrr %aiii|ih'%. IIn* hull li\|iiitlM*%ift is llial imi diffrr- 
cfiei* r\M% III IIm' r tiliuii^ irf iIn* |Mi|>iiblNNi« fpNii hIik'Ii IIn* %ani|>k*% ;irt' 
(Imwii. 

TIn* iiMiliaii cif iIn' c*«hiiIminiI ( |«nii|I\ riiiikc*(h ni r n.* * ii; * ... 9U 
<iliM*r% aliiNift U ciikiiV. It'll. TInji |In> «aiii|>k*» urc* M*|)iir.il«Hl. iiinI c*iM'li inn* 
it Irt'iilcnl c'luidh a% ck^ncrilN'tl uihI iHii%tnil«Hl rurlk'r in IIn* cum* <if iIn* 
Mrclhiii T«*iit Uw Two liMli*|M*tMk*til Saiupkm. Tluil ift. f«ir «%ic*li M*panilc* 
kiimplr iIn* iHinilMT cil Htirc*« uImhc* iIn* jiiiiil nirtliMii i« c^niiiI«hI. mihI iIn* 
iMimhrr cif Miin*« wliicli fiill ul m \m*Um iIn* nircliaii it ctiiiiil«Hl. TlN*t4* 
%iiliirft iin* IIn-ii c*tilc*rcil iiilii 411 fl ( nm% ) - (• ( cnikiiiiitt ) c^ M iHtp p nc^ lulik*. 
A dii Mffuin* l«*%l it ii|i|iliiil iitiiix iIn* fpiN^ml fiiriiHiItt Inr c*lii i«fiian*. 
lliit prvKttliin*. c^xiMtlctl fni|iii*iH*iit iiNttt In* ciifii|iiiliHl TW Hifnifkain'c* 
III iIm* cililiiitMHl t4 dii t«|ii4n* can Im* n*iid fnmi Ikr pMWvil li >k* «»f 

llir DMrilNilimi cif (lii S«|iMrr mI J/ N \) (C I ^ 

Tiilik* III prc*t<*filt iIn* ckila fiNr fcNir Mini|>kt: 

T«M9 Sc#r«t i#c«r^«^ $99 ^•nr iiii M iWi m t O^ttm 




TIm* lutnl inihiImt iif ulNmiilkim it HI. TIm* mrdton it iH. \V\h*%% wi* 
nrrmiKi* llirtr ckilia in m c^Nitiii||nM*> liihir mid Mwitpiili* iIm* rtptxlrcl frc- 
<|iinK*^. wr \mM* iIm* iImU |in*trfilrtl lit T«l4r 11, 
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WHk thptr JaU, wr proeml dimtly tf» c*aiii|iiitr cln ^^iiurc* at wc* 
dbi in Ifer «ar of tw(i md^prwirnt Myh i Hm* <ibtaiiM*ci iliic c>f clii 
■fMire caiaatertrd Inm thb lalilr m 8.2H Tlie tiiimbfr at dcy^fr^ <if frcvchMii 
H (4 1 ^ v3 1 ) ~ 3. Tlir tabled swtne vt clii M|iiarc* rr^fwrtl Uh 
MMcttncf at Ihr 5 pt*r cmt le\rl iit 7.M5. Ttie ohatr^iHl %ak«« u %li|i;litlv 
mmrr thtm thit. and we may conclude itMt an mer-all (two-tiialed ) differ 
ever eaMi among the foiir KHMipt. 

flwdt^lfMw TmI ftr Mlwt Mm fwm hni^pmnimilt SMiplM is 
itraif{htiarward extension of the clii wtfmtr text for two ifKlependent sani 
pies. In general the test is the sannp Inr two independent groups and 
three or more gmips. 

The null hypothesis is tliat the C MM^4es of fre«|iiencies or proportioti!>i 
ha%-e come from the same or klentlnt pnpiibtions. This hypntliesis ma\ 
be tested by applying the general Ammuh for chi scpiare where each 
observed fiequency, or proportion. lm» mhtracted from it the <*«irrespotMl 
ing expected or theoretical fre y iw^ 4he diffetmce is M|uaiMl and then 
divided by the expected frequency: «hi aA resulting quotients jvr summed 
to vield the valne of chi square. As m Itn* chi square test iar two inde* 
peii^Mt samplvi^ the data ( f rvcfamictes m p iap n r t ions) are nsrtered into 
an Jl (nM») > C (cokimns) rontinmaic\ table of dinct^ categories. 
These dnin may be expressed in tenm of nominal or ordiHai icalet. 

For mmmfkt, suppose that we are asderested in testing «iar differences 
in children* 1^ scores, grade level l>y gsade level, in one ekmrnalMry icliool. 
We can set up a contingency table with sis eolumm rrpr g sami n g the six 
grade levels in tlie ichool. Then we can categariae the poasllde range ivf 
IQ scores m increases of ten-point intervals, and enter seven such cate- 
gories in the rtrns iif tlie table. The blank table miglit look Ubr this: 
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T«M# 11. A MmIi Coii«wi9«fKy T«M« lor IQ Scorot by Uv«lt 1-^ 




U rr wr wp wovkinft«Mlh a vrfx mrge contin9Mr> Uriib* which is 
6 ' II »ia». Hie dcgr < ;c'»i rf frmdiNii air (6 1) (7 1). «r m There 
are 42 cells in ^ laUe, mm for th4' <iWrved fre<{iiem*\ frCOfwd in each 
of them, an exfrcted freq«mc% wmtst lie calculated throiip(h manipulating 
the totak in the margint. tiMmmk\\ tlie amount of labor it incfwed for 
a problem of this tiase. biit the paacvdure remains the same as tint pre- 
vioiish' presented for two iiit pi'nAMt samples. 

For this use of chi sqiiair. six separate independent samptwi are being 
tested simultaneously for an over-all dif erence bHwern tham The test 
is two>tailed. It will enable ttie ia ^csUga tor to detenalMe the lignifa nm i 
of any difference detected. U«it of votnM' wM not, itself, indicate tlie 
location of that difference. 



Up to tiria tiriut, we Immt been efuKYnwd chiefly with tlv effect of a 
>ii>g|e tn drpm ^wt variable. We ntm u— idiT the prubkni ai iblrnuiuing 
Ar degree of si— k si uiu MS or coocmIM variation of two variables. Tlie 
daU under coMdnration cwMt sf |MBi of measurements. For example, 
the data may he measuees of both baifN and weight of chtdrrn, or meas- 
ures of both int a li gfnw ? and scholastic performance. The — ntial feature 
of the data is that one o h s s swiio n can be paired with anoHsrr ohaervaticAi 
for eac4i member of the graiip. This tape of data has two doaely related 
features: and paadiction. Correction is conomed with the 

degree of relaUan betw een two variaUas. Fredictkm is rasicemed with 
estioMting the aalant of one variable from the knowledge of another. 

Ihm ipmmmm tkm$k AmmMm CMffWtut was one af the earliest 
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Rrs. <fr • flu- PrucHtioncr in Educntion 

i»tutistics hasrd on ranks and ^n<- of the Ix^st known today. It is what 
is known as a ''prodnct-moiiiHiit ct)rrdation cm*fficu'nt and is similar 
to tlie esteemed (and more df#uidrr) Pcif^^)n correlational method. And 
the SfKarman formula enjoys .iiiitmnal advantages of its own. In many 
situations where ranking metuMs ^e used, quantitative measurements 
are not possible. For exampk evldrc n max Ik* ranked l>y teaclurs on 
s(Kial adjustment, friendliness. <«r m^m^ (jiuiitatixe fai^tan. In such cases, 
the data are ctmiprised of sets m ^im^mmd iMMwhtTs (first. *«t^rutid, third, i*tc.). 
Or, if the data are expressed tavk at earAnal numl>er\ < one, two, three, 
etc.), tliev may lie used dirvc^ 

(>H*Hieu*nts of (.*orrelation ntiix rf 4bimiuaII\ di*Hn<Hl to take tlie values 
}!,(), ;uul 1. These indiualc\ ii tl»f n^if r giv<»n, a jKTfeet )K)sitiv(v 
neutral, aiul perfect negative w^Mttmm ln-t^ft^ tlu» two xariahles. And the 
ilegn es of ri>lation are alwaxs a dt-eimal fraction of 1. 

The Si^t^jruian formula cbvr^ViitM u» tnakr .1 list of the ^il)jet*ts. Next 
to (*aeli sul>iect\ entrx , his riM^ •w Hm \ variabh* is rei^orded. and his 
rank for the Y xariahle is rt-t^i^^rtiwi Tf*« (Mlrrene«* lK»tW(vn these two 
ranks is recordt^d. This di#tw«rr m Isilueii 1)\ tin* letter (I. Then 
r.ieh (I is M|U4r(Hl t.> ol>tain a vfttmmm «d( «/ t'Vrii the d' (^>hnnn is sunnneil. 
W hen we kiHm tin* values ol V hmwIm > ni mky^H:is ^ jtid r/*, tlu* c(H»ffi- 



ci(*nt of (orrelation can In* ccmpr^rd. Hh* kwi nu la cnvluins tlu*M* steps 

mv times fkr sum of d~ 

i'itrHiricnt tme whole numhr- , . ^ x-. , 

' mmmmi'f m sample {\)^ ^ — i 




• mm «p # t It 

• 4P 4ii • • • 4 

• m \^ m # » % m 

• m ^ f « 14 

# ^ t 4 



Statittkal TeMs 



By way of illustration, one investigator was interested in the strength 
of the relationship between achievement in reading and achievement in 
arithmetic. The scores of ten eighth grade students in both areas were 
selected at random. The results are shown in Table 13. 

There are significance tables available for the rank correlation proce- 
dure. In this case, the coeflBcient of correlation computed from the Spear- 
man formula is .722. Reference to a table of significance values shows that 
for an N of 10, this coeScient of correlation is significant at the .Ofi level. 
We nmy conclude that proficiency in reading is significantly associated 
with proficiency in arithmetic. 

As we have seen, the selection of an appropriate statistical test is 
related to the research conditions which determine the operational defini- 
tions of the variable involved and the questions which can be asked of the 
data. Different research conditions call for different statistical tests. The 
nature of the population from which the sample was drawn, the way in 
which the scores were obtained, the kind of scaling which was employed 
to express the scores— these are among the considerations which must be 
taken into account in choosing an appropriate test. 

While this chapter has not discuned all of the available nonparametric 
tests, the eleven which have been presented are sufficient to permit analysis 
under most conditions found in school administration and teaching. The 
investigator, of course, must exercise wisdom in selecting the test and in 
applying it to his data. No amount of statistical sophistication can substi- 
tute for good judgment. 
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To say that M resewch profects should eventual^ be rsAsced 
to words aad symbols WQ«ld be to nin tlie risk appMring 
pedbntic and tediow. Crrtainly, the custom nt prcmtorel)^ 
ruiliiig ideak and exprrirtices into writing is siehoolish and 
fru p u ntly iliMXHiraging. Formal verbalization of ideas and 
findings wliich have nut been refined through a i>eriod «tf in te-Scc tual 
gestation anHmnts to little more than the embalming of primiMMr coneepts. 
No, not aM rmearch activities should find their way into reports. 
Some slKmUi he saved for the dilettante pleasure of simpK hn^g fun. 

But it be a mistake to heheve that written accwH^ aiid reports 

have little valne. I1ie nntiire of tin' vrwarch process makes it ^rticubrly 
im|xirtant lo record the (piestion beinf; investigated, the steps hntug taken, 
the decisions made, and the divovefies encountered. The* ^nnmtific oh- 
servrr regaflis his work as "public/* He welcomes checks and wpct i tiniis 
of im work as complimentary and as additions to the knamfcilge lie is 
seeking. He strives to repnl lus work in sufficient detail to endUe himself 
and others to> pursue adchMonal related research studies. It is Afmigh this 
kind of repnrt that reseaitsh becomes cnnwilative and uncertainty m rediicn^d. 

In addition, tlie reisnrcher genenJk' finds it quite necesmr}' to keep 
an informal notebook. $k«rh a iiotelionk m used to record immeAntely data 
on ohser>'ations made. Because the hnHam memory is notoriomK^ fattble, 
the notebook liahit is sonn niislnred as an indispensable convenience and 
is seldom rrgarded as dmdf^r\' Manx rspbralory and prrnneseaich studies 
are not taken beymd the notehnak phaar. But even in these cases, notatHiiis 
may later prove to be extrrnn nl y suggeattve far other studies. 

Custoaaarily. a report on cowip U i id research, when it appears as an 
article in a professional journal or wiien it is flnd for the use of colleaffnes 
in cnrricuhnn imfrovement pssgrams. follows an outline snch as this: 

K Ptm mn^ iam of thn PmMmn wMi mirfi fhn Study Wm^% 
—Review nl tlie litemfenre, if appoprinle 
—Review of prior stn^Ks, if appnprinie 
-"-Summary' of local iwms, if appsnpsinle 
--^lear sMement of sninlion being ■■■fCht 

— Oescripiian M pnpniaiinn «r samples uaad 
-Objectives or hyfrnthew^ l>Mg tested 
— Instrnnw iitatinn and Attn nnMeciian mii iii u ds 
— SmMnary taWes, wha^^^^jpsnprinte 

— Vtethnds of asimng anM. inclndinf( slnMscal tesls and ibniBinns 
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III. llMultS 

—Presentation of findings, including levdi of statistical sipiiicance 
—Logical analyaes of major findings 

IV. DiscMHion 

—Reference to objectives or hypotheses hang tested 

—Ideas on extended meanings of the findings for additional studies 

— Befeimces in standard form for bibliograpliical search 
— Annotated references, if desired 

Although this ovtkne is extremely useful .in econonnically reporting 
research findings, it i m io it unately tends to produce a report which gives 
■ninlended and inaccurate impressions of wliat researck actually is. Pub- 
hihed reports strongly suggest that research projects hnve been carried 
Oik in a sequential faaliion and that they always resi.lt in "significant" 
indings. But the research process, in the doing of it, ahnost never folnws 
the neat sequential pattern suggested in the reports. Furthermore, ''faikwe" 
to achieve significant findings or to verify predictions nMde is the usual, 
not the exceptional, end resuk of research studies. 

The reseaivh process would be much simpler logically if the researcher 
could follow a prescribed sequence of procedure, each step presupposing 
the completion of the preceding one. In practice, hcMPever, it is frequently 
necessary to take various steps out of orcfer-sometimes to backtrack^ other 
times to compiess two or more steps into one, and on other occasions 
simply to omit one or more steps. Also, the iMomch process invokes 
many additional activities which are rarely mtw^Hmmmd in it Mmnh mparts. 
For example, methads of data collection may he ncaiiy rainriioriwd, 
skipping the mare involved deciskms ohont the bad of data needed and 
the activities carried out in devebping and pii iiilin|i, the data coHection 
instrumenlB. 

The ni^pression that reported reseaseh stariirs ape invariablv ^'success- 
lal*' may hare an even more unfortnaale coasa^pmce ihaa fLt impres* 
sian that Aa* iesearch process folk>w» a p i eat i ihtj seqaence of steps. It 
aaiy he trar that puMislied reports of leseareh otedbes in education have 
tnded to io«eal only the significant and hore aippiesoed the nonsig- 
aAnnt. If oa, educational research has batn loas than halfHreported. 
Maips we alow our research a c lisi H a i to h ac oow t confuoed with our 
fpnaral worisiday obsession with "suoDf»« and "faihar/* These two words, 
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wliicli luivc takc'ii on a inoml ovrrtonc, arc out of place in rcfrrcncr to 
research. XowliiTr is tliis nnirr apparent tlian in relation to tlic riwarcli 
activities of American indnstrv . In nianv iniltisti ics, research projects that 
result in new iuul profitable discoveries are ilistlnctly in the niinoritv. The 
majority of the pn)jects do not work out as intended. Iwit the\ are care- 
fnlly preserved because knowing where the |X)tential blind allevs are is 
greatly ini|K)rtant. In a very real sense, no carefnily constructed research 
project can be worthless, regardless of its results. Kverv project can add 
useful information to our knowledge and thinking, .^nd the nonsignificant 
stndic*s can make a n(*i*d(*d contribution if they are presented in written 
form so that we and others can tliink alMint them as we plan for similar 
studies. 

It is (piite riMsonablc to expect that each individual scIuh)! and each 
individual scIhm)I svstem will one day keep a library of research reports 
completed l)y its own staff. As more and more school svstems alliKati* an 
anmial budget to snp|)ort IcKal research, it will 1h» increasingK ex|Kvted 
that the outcomes Ik* visible. The NEAs Project on Instruction re|>ort, 
Schools for the Sixties, makes this recommendation: "The National Com- 
mittee has reconuniMided that each school system allocate a definite and 
adecpiate proportion—at least ouo |x*r cent— of its annual o|)erating budget 
for the sup(x>rt of research, planning and development."''' When this is 
done regularly and when research activities Ix^come a budgetarv line 
item, school |x»ople, lK)ard mendx'rs, parents, and colleagues will develop 
a regidar hunger for l(K»al re|X)rts which indicate sc^titiik pFivwth in 
sc*hool development and staff improvement. 



The literature on teaching which is found in college textbooks 
and many professional journals for the most part does not 
include reports on research. The conclusions of research studies 
and their implications for practice may be referred to, but they 
are mentioned in quite general treatments. Such general dis- 
cussions are useful in undertaking to build a background for a given 
subject, but they lack the detail and specificity of original reports. Par- 
ticularly the administrator who wishes to build a background in current 
research subjects will find himself in need of different and more special- 
ized sources of information from those provided by the customai v literature. 

It is entirely reasonable to assume that few administrators and teachers 
subscribe to specialized research journals or have knowledge of their 
sources. This should not be surprising because traditional programs of 
teacher preparation, and much current in-service literature, are not focused 
on the understanding and use of original research reports. But for the 
person who would become involved and develop skill in conducting his 
own research, acquaintance with the small list of journals most likely to 
contain papers on the subjects of his interests will be most rewarding. 

It is very important to keep a steady stream of new ideas flowing 
into a research project. Often tliese new ideas come from browsing in 
specific reports of what other researchers are doing, liow they state their 
problems, treat their data, arrive at conclusions, and the like. Becoming 
overwhelmed with reports on what is being done should be guarded 
against, and a search of the literature should not be carried too far or it 
will go on forever and serve as a complete bar to action. On the other 
hand, even the wise selection of pertinent literature is impossible unless 
the investigator has knowledge of the structure and sources of researcli 
literature. The effort to acquire this knowledge will repay itself handsomely 
in saving time spent in the library and in selecting journals for private 
subscription. 

In regard to the literature, there are two important and feasible goals 
for the researcher. The first of tliese is to find whether the problem area of 
a proposed research project is being investigated elsewhere, the names of 
principal investigators in the area, and the kind of investigations being 
pursued. The second goal is to acquire reassurance in broadly questioning 
the practices of schools and teaching. It should be borne in mind that 
findings coming from research projects in education and in the behavioral 
sciences in general are seldom conclusive or final. The extent to which the 
generalizations k,*" even the most carefully conducted research project 
can be applied to a specific classroom per se is probably very small. The 
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foregoing two statements suggest that it is preferable for the investigator 
in his own situation to "try" experimentally and check out others* research 
findings before he adopts them. This sort of replication is one of the best 
and most needed contributions to the broad field of research on teaching. 
It also represents one of thr finest uses of the research literature. Knowing 
where to locate and how to use the available material can be a desirable 
addition to the educator's background. 



■OOKS 

Books and textbooks provide basic material and a general level of 
information. They provide one of the first approaches to a new speciali^a* 
tion in order to gain a feeling for the whole field. There is no simple way 
of finding all the books on a given subject. The most immediate place 
to look is the subject index of a good library. 

There are a few books which attempt to organize material on methods, 
research areas, and techniques of searching which seem generally useful. 
The titles u'hich are cited as references in this volume are not repeated 
here, although they, too, may be cpnsidered part of this list. 

Carter Alexander and Arvid J. Burke. 



Fourth edition. New York: 
Bureau of Publications, Teachers Col- 
lege, Columbia University, 1958. 

The b(x>k a) brings the user up to 
date on hundreds of sources for locating 
educational information and data, and 
b) makes revisions in methods and ref- 
erences contained in earlier editions 
which save time and energy for users. 
Included in the book are two main 
headings: 1) Basic Techniques of Li- 
brary Utilization which includes chap- 
ter$ on locating books and periodicals, 
using the indexes, library reading, and 
note taking; and 2) Special Applica- 
tion of Library Utilization Techniques 
which includes chapters on book evalu- 
ation, government documents, refer- 
enc*e b(M)ks, cjuotations, statistics, bio- 
graphical information, etc. 
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O. K. Buros, editor. Tke 
KliMlftIi YcwkMlL Highland 
Park, New jersey: The Gryphon Press, 
1959. 

Lists 1,957 commercially available 
tests published during the years 1952- 
1958. Also included are over 6,000 ref- 
erences on the constniction, use, and 
limitations of specific icsts. There are 
tests of bookkeeping, Latin achieve- 
ment, English progress, nursing, sensory 
acuity, coordination, tales aptitude, 
handwriting, etiquette, sex knowk^dge, 
health, religion, honesty, and many 
other skilb and aptitudes. 

Stephen M. Corey. AcMm lUtMrch lo 
Iwpraii SckMl PMIcits. New York: 
Bureau of Publications, Teachers Col- 
lege. Columbia University, 1953. 

Addressed to practitioners in educa- 
tion and stresses cooperative research. 



Htf Ht s*Hf* h l.ttrrahtri 



It liNiltiiiis ti stiiniil.itin^ i liaptiT «ni tv- 
MMrrli in «>«liic .itiiNi. m>\it;iI ititiTrMiii^ 
tirroiiiit% «»f ;i«lMNi rrMMrili pr«»|<*«'l%. 
.mil jii r\« i llriit HO iU u\ l»iMiof(Mphv. 

N L . « <iil«>r H— •! lit- 
iitarrll mi TfarM«(. A rr«»(4cl «»f ttn* 
Amrrii^iii Kilm utiiMial H(>M4*uri ii ,\%m' 
iiatiiHK \KA. (lliii-ago: Hand McNally 
C!«Miipatiy. niAl. 

Tim )hhA, lim h(M*n iMlk^il tlit* tmMl 
impirtani iipntrib«iti«ifi l«> I'diu aliiinul 
rrM*ur( li m<>tlHMl(»l«»f(y y«*l m4«l(*. It cim- 
ttiim 23 M*<-tNfn% written hy 31 ^uitlnirx, 
iMcli «if wImnh i% a Kpi^t'iali^t in )ii^ p,ir- 
tit iihir hi'M. TIm* )nnA is «irf(anixi*<i iiitii 
f«Hir \wt%: Part I. 'TliiHirtlkal Orkiita- 
tiim^.' Part II. "M«llMMl«i|«>Kk's in R<*- 
M^art-li mi Tisuhing.'* Part III. "Major 
Variat>l<*!ii and \fvn% «if R^M^arcli «in 
Tc^ac'liing.** ami Part IV. *'IUnM*art'li tm 
TcMiliing VarkniK (track' l^*v«*U mimI 
Siihjc*it Matt«T»/* llw I.2IH pti|{i*« «if 
tht* huHik pr«*5M*nt a siilBcktitly umpn*- 
iMfitivc* rangi* itt haik* nMt«*H4il t«i my^c* 
flu* int«Tt*!»lt and nvnh til mutt p*<ipk* 
intc*mtcHl in rrti*an'h «in ft'iM-hing. 

Jt-nww ). Ilaiitman. «*dit«ir. Riiwtk hi 
Alt riilKliB. Ninth Y«*arlMMik. Waih- 
injfitm. irCs Tht Natinnul Art EdiKA- 
tkm AiMKiatiiin. NEA. IWH. 

A ("impAbtiiifi of itimpk*trd mrttrc4i 
BtiMUn in art rducatitin. Fivr main di- 
viftiimt— pMliMiipliical-ptyi*linliif{k'al rr- 
learch. frwtmnh and crrattve hrtuivicr. 
frtrnffti into Irariiin^ proem, infvfYi 
and df u rip t ivr wtr a g i h. and f f gat ch 
NifiF pninmnB oi fpacnniK nanmrappcci 
and ruerptiunal chiMmi-arr carrfuHy 
rxpkjfvd by cnititaiKNnf auHinHtirf in 
thr art Md at both the high ichnol and 
univmlty fevdi. 



(!. \V. Iliinniciitt and William J. Ivrr- 
MNi. rditiHT^ Rtuvarck In tkt Urtt RV 

\rw Yiirk: Harper h HnitlMTv IM8 

(jNit.iinx xtiKlii'^ of varying li'vrU <if 
M irtitiKi «|iiality di'aliiig with rrailin^. 
writing, ami arithim'tir. Tin* wi»rk« in- 
rlinh'd witc «'i«»M>n fur tiM'ir Ki^ifi- 
ramt*. infliicnn . ami imprtlamr in tiM' 
fk'M. AiitlMiri «rf tin* itiidii'x arr Wfll- 
kmhfvn piHipk* with npiHial intf*rt*it» in 
thi^M* aroas. 

(•ar(lm*r Kiml/«*\'. «f*Mcir. Handknak al 
SncW riydmlav. ViA. I. TliiHin and 
McllMMh** Camhrklgi-; AikliMin-\V€^> 
k*y Piihlixhing CI<ifrp.4i,' Imv. IM4. 

T!n'n' an' nim* (*hapt€*r« di'viitiHl tii 
rc*iM*ar(h mt*tlHMK. Like* hamlhiMikx in 
many «if thi* ack*mt^. tlii% «pim* %% brgc*. 
tfilmkal. ami inf<irmativfv 

Phi Di-ha Kappa SjmifMlMn an EAh 
caManal BmaiA RkMiinin|{t<in. Indi- 
ana: thi* Fraternity. 

Thii wtf^t% cif vc4umrft imliKk's f«jr- 
mal papm and suhir«|urnt diicustiom 
i'lanpfMnfi twtMlay symposia umklrr- 
ing diii 'rrnt atprrti of rducatkmal rr- 
wtM%M. Thr Ant biMik (1980) draft 
with thi* itinmt stafr <if drvrkipmrnt 
in c-diMatkaial n^srart-h; the irt^md 
( IMI ). with n^M-an-h design and anal- 
ysfe. 

Ciipk^ cif thetr hiinkt air pubUfhrd 
vear and mav he obtained by 
writing Hi Phi Deka Kappa. Inc.. aih 
and Unliai. IMiwniinglnn. Indiana 

Phi Ddta Kappa. RaaiaRll llnAn In 

EAKSrtan* MiManlii^on. Indiana « the 
Fraternity, 

Thete ftiidlet are Mibfect-author in- 
deiei and rrtrarch methodi Mhlkigra- 
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]y\uv%. Di^Mrtation^. n*porti». and A(*ld 
xtnclh'x imcItT way or compk*t(*cl at col- 
\vnv% and iinivfrsdtit*!^ in \}w U.S. and 
C'anada art' cla^kkifird by siihjoci nndrr 



gt'nt'ral lM*adings. 

Availahlt* in mtMt iinivorsity and cm))- 
Irgf librarit*s. Nrw editions appi*ar 
yearly. 



iNCYaomiAt 

Tlir uvxt lv\v\ of s|Mriuli/atioii for most of tlie scinices consists of 
tin* various eiicvclo|M*cliasi wliidi arc rxtrnnidy useful for accpiiring a 
it>ni|>rrlimsiM\ yrt drtailcHl. view of a given field. Only one encyclopedia 
cente ring on educational researcli, wliicli is sponsored by Tbe American 
Educational Rc>K*arcli Asatociation, is included in this list. It provides a 
convenient oiH'-\(>lume source book of unquestioned autlKirity, providing 
ri^*arclM*rs witli recent developments in their areas of special interest. 



Ri>bi*rt KM, t-tiitor. Eacyrtipiih •! 
r iB Hi li— I RtMNk Fmirth Edition. 
Tlie Macmillan Company. New Y«)rk. 

TIm* t*nryclop(*dia presents a critical 
( Valuation. s\^tlM*sis. and interpretaticm 
of all tin* p(*rtiniiit rrsearcb— early as 
wfll a% rei'iiit-on a variety of subjei*ts 
^prrad over tlw* tiitire range of educa- 
tional topics. Four edititms of the vol- 
ume have hec^ published: 1941, 1980, 



1960, and 1969. Tin' AuM-rican Educa- 
tional Rt*st*arcli AsjuKiation has hwn 
responsibk* for tin* publication, with 
two diff(*rent educational k*aik'rs serv- 
ing in tlK« (editorial capacity. Each new 
volunrM* updates the material found in 
the other tmyckipedias. At tin* end of 
(*ach topic, tliere Is an exhaustive bibli- 
ography of both b<N>ks and pi*riodicals 
which am ctjnc(*med with the* specific 
topic^s treated. 



urauniu OMDn 

Beyond encyckipedias, one comes to the general guides for seledkm 
of literature. Tlie guides generally classify titles IIr^ ^ ' •metimes in- 
clude reviews and criticisms of the content. ^ctimet are 
descriptive and frequently are a valuable he^ lu c ssr ig availiUe 
literature. 



Mb. American Library Association. 

Chicago. 

Formerly the SiilMrriprion Book$ Buf- 
hiin. the perkidical was combined with 
another publication of the American 
Ubrary Assodatkm. BooUUi. in 19eW. 



The puMicalion is used mainly by U- 
hrarians in evahialing new books. Up 
to lose, the periodical gave critical re- 
views of various referaioe books, e.g., 
encydopediaa. indexes, etc. In this new 
publication, other books are also in- 
cluded in the critical reviews. 



Ill 



The Research Utvratun' 



Puhlishcd twice a month Scptemher 
through July and oncv in August. Singli* 
rupirs may be* uhtained by writing to 
tho Amcrifun Library AsscKiation. 150 
K. Huron St.. Chicago. Illinois. 

CMMvliMvf Book iKdfX. The H. W. 

Wilson Company. New York. 

A world list of books in the English 
language. Information about any hixyk 
is available through three sourcH?s in 
one alphahi*t: author, title of the book, 
and subject. For a b<iok for which the 
title is known, information may be ob- 
tained through the publisher s name. 
Information is arranged under names of 
persons by: 

1. Works which the person authored 

2. Works of which he is joint auth<ir, 
editor, or translator 

3. Works about the person. 
Under names of places the order is: 

1. Official publications by depart- 
ments or bureaus 

2. Worb about the place 

3. Societies, institutions. 

Published every four or five years, de- 
pending on the magnitude of the num- 
ber of entries to be indexed. It was first 
published in ltt)2, and the current edi- 
tion is a 1059 publication. 

Etmc^Om ladtx. The II. W. Wilson 
Company. New York. 

A ctmuilative fubfect tide index to a 
selected list of educational periodlcab, 
books, and pamphlets. Near the be- 
ginning of each volume are listed 
the various periocUcab from which ma- 
terial has been indexed. The first vol- 
ume was published in 1932 and con- 



tained material written from 1929 1932 
(3 years). At prcsmt. a volume is ac> 
cumulated every tWee years. Small 
papi*r-bound copies of current months 
are available so that the reader may 
find the latest literature if he wishes to 
do so. 

NEA Catalo|»c of PoMkalioM. Nation- 
al Education Association. Washington. 
DC. 

Lists publications under the headings 
of the NEA units responsible for pro- 
duction. Incorporated abo are alpha- 
betical and subject indexes. In each 
catalogue*, the publications are listed 
first by number and then are r(*fem*d 
to by that number afterward in the* 
indexes. Entries in green print are new 
issues. There is a research division of 
NEA. and its publications are included 
in the listing. 

The recent catalogue may be ob- 
tained by writing to the National Edu- 
cation Association, 1201 Sixteenth St.. 
N.W . Washington. D C. 

H. W. Wilson Company. New York. 
1957. 

An author and subject index of arti- 
cles taken from many American peri- 
odicals. These publications are listed in 
the front of each issue Provides a wf4l- 
balanced -lect*^ copula* * h- 
nical — "jir — iiting aN tik im- 

ports scM iniiu ti*chnieal. and sub|ect 
fields. Current issues of Readers Guide 
are paper-backed publications and are 
prcxiuced every two or three months by 
the H. W. Wilson Company. 9S0 Uni- 
versity Avenue. New York. New York, 
Available only in libraries. 
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ABSTRACTING ANB 

TUvsv journals arr one of tin 
is siH^king iiiiorniation on spccifif v 
siiviiii; uuili(Kl of kivpiiig np w itii i 

DisMrrtalion Abstracts. I nix tTsitv Mi- 
irofilins, Ann Arlxir. Midiij^an. 

A iiKuitlily c'tmipiliition nf alistracts 
of (Imtoral dissiTtatinns siilimittcHi }»\ 
uiorr than 115 ctNtpiTatin)^ institutions. 
I-Iatli issnr consists of a principal S4't- 
tion. an autlior indrx, and a siib|Vit in- 
drx, TIk» principal srrtion contains tin* 
abstracts, arranged undrr the sid)jcct 
t atrgoric.s assigned by author. The iil^ 
phalniical hst uf categ<»rics is given n 
the tal>N' iif contents. Anth<»r and siU 
jc»ct indcvxes are cumulated annually -ami 
inehide "sec als<»" references to mo 
specific or closely reflated sul)j(*it 
^A|so. a xcTox copy uf any dissiTtati« 
tluftJjas Ixvn completed in the Unitt- 
States c;^*i Ih» secured fn»m Unixersit^ 
XhVrofilms. 

For a snbscripfh*q^)r for a \vru\ copy 
of a disscTtation, \ynte to University 
Microfilms, Xerox Corporaticin, 300 
2t»el) Road, Ann Arbor» Michigan. 

EdncatiM Abstracts. UNESCO Puf)li- 
cations Center. New York» New York. 

Kai'h issue of the puf)lieation deals 
w ith one topic only. Abstracts are listed 
by country and are sometimes written 
in tlw native language. Many of the 
issues trnter on education in other 
UHintries, curric ulum, and educational 
methods. 

Published every month except July 
and August. Single ctipii^ may be ob- 
tained by writing to the UNESCO Pub- 
lications Center^ 801 Third Avenue» 
New York, New York. 



" iMG JOURNALS 

f rs for the inyestigator who 
liey also pro\ ide one time- 
< arch emphases. 

1 «ffical Abstracts. Amerii an Psy- 
.d A.ss(K'iatioii. Inc. U'ashing- 

• c ts of \-arions studic^s Inking 
• III in all major areas of psvchol- 
- rt»i>)rted. Problems dealt with 
* tin- nerxous system, learning. 

\^ <nalysis. ]XTsonality. trime and 
t*i t *my. mental testing, and others. 
^ ' »hniie includes one calendar 

d is indexed by subject and 
i^w Mondily editions are also ayail- 
current studies. 
>hed mondily widi two issues 
riilxT, Copies may Ix^ obtain(*d 
ting to: The American Psycho- 
^ ii Asstxiation. 13,13 Sixteenth 
Vfrt t. X.W W ashington, D. C. 

tedohpiBal Abstracts. Leo P. Chall, 
editor. New York, New York. 

The purpose of the piTiodieal is to 
present abstracts regarding, in order of 
priority, 1) Mxinlogieal [XTitxIicals and 
IxNiks; 2) piTi(xlicals and bcniks biMring 
on s(X*iology; and 3) nonsociological 
piTimlieals containing significant, ap- 
plicable material. Abstracts are listed 
Mbjeet areas by author's name. 
tTi<idical was first published in 
^ (is now issued five times a vear 
«• uary, April, July» Oetober/and 
\o\ ember. Single isuses are available 
by writing to the Editorial Office, c/o 
I^o P. ChalK 225 West Seth Street, 
New York, New York. 
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«¥IEW JOURNMS 

Revif-w jiwii tJkspemsMe in ewmlkmmL rrsearchers to have 

soMe idnt of ^mm ^whl «*c()niplislied akMif^ -aaams lines of investiga* 
tion. Tkrse s(>in4^> jpnrra^* irovide more cfiravM^ information than the 
ahgtradwig joit rn a i 'trrnnuf vsciissing leportN ir trmvs of their theoretical 
settings 



chaAogical Assoc^ia^m \t\ ^ ishing^ 
ton. D C, 

EJevoted to artirV^ ^f tb^ al sig- 
nificance* to any > }i Ki**.m' en- 
deavor in psychoivifi Hmtmmmm 
penodical does no re- 
search but makes e . • ^ ^^"•v this 
researc*h is inchai^ m thc^)- 
retical article whmch ^ttman^ tti inte- 
grate several rekHiv^ ivn$mnJ^ mAvs . 

Published binwiMilih .md Available 
by uTiting to the mrm^mm Fswholop- 
ical Association, w... rPti^ "sixteenth 
Street. NAV., \\mkmm^» .#C. 

Review ciEdvc^tarf^Mtf^ Amer- 
ican Educational ReMaa^ ^Mwiation, 



\EA, Washmgton, D, C, 

The purpose' of the Review is to re- 
port major reseaix h findings during a 
designated period, organized by areas 
of inter€*st. Significant studies are iden- 
tified, summarized, and critically ana- 
lyzt*d. The more active fields of educa- 
tional research arr viewed every three 
years; the less active fields are included 
in alternate cycles. 

Published in February, April, June, 
October, and December. This journal is 
a "must'* for researchers in education. 
Single copies may be obtained by writ- 
ing to the American Educational Re- 
searcli Associaticm, National Education 
Association, 1301 Sixteenth Street, 
NAV., Washingtiwi, D.C. 



MBMN JOUMiALS 

These joumak an* "^tie Mwury source of complete and full-flavored 
m i l rial in "repovtrd'' nmewM pmjects. Commonly, they* contain descrip- 
tmm and dtacuHions of i i w iifiiques, designs, aHvirta, and sampling 
iMiftods used. They ^mad i^«t to be unduly compmaed, and contain 
snflhricnt detail to maim ' wmMc critical reading and replications Ih 
iMtk iiJled pmfie Thmm .mmhIs provide the richaat type of prinnari 
aaipnitanor iiilh the Htmmmr on research in actian 

^lie jOMnb kstedliBPe ^m* fpmtped in three major Mh-chMriftMliom 
e^Ht^^aian, paychflilaip^. aa^ t^m^n^^tf^ 

Em^ *mm i his journal mmiit * <iehnt in Janu- 

^^t^m^^^^tt^^Ktt&tmm^ ^ 1^^ 1^ paMtahM wigiaiif repoiti 
Ani' van Edacatannd Wkmrmm of experimental md IWatvtieal studies 

ctat ), NEA WaaMnglan in education. fc piai ''j inchaled are 



Jt :rrh for the Ffm uu tm rn er m Eduction 



Of 



L-omplete, CHmtaining infci 
pwpiMes of the investigatkm, 
jmd procedure, description of tl 
pie studied, results, diseussskm 
and charts, and conclusions, 
of books on research are inc 
each issue. 

Published four times u year 
c*opies may be obtained by 
the American Educational Resei 
sociation, 1201 Sixteenth Street, 
Washington, D.C 

CaMorate Jmmal of Eim^m 
scaick California Teachers Ass<k 
Burlingame, California, 

Reports on: a) city and county i 
research pertaining to curric 
guidance and counsebag, evaluiilH^ 
supervision, and finance; b) digested •«! 
theses and dissertations that have pm* 
tical application; and c) studies ibmi 
present novel, but tested, approiHlHtf 
to the solution of educalional problem 
Certain articles may abi) be hnmi «. 
Pmfchoh0od Abttracts. 

PuhWted five times a %«ar: 
March, May, September, and 
ber. Single copies are available wfm 
request from California tmmrnaimf i 
cahtmal Rnearch, 17K 
Drive, Bnrbigame, Calif«imia. 



EducaMw Association 
Toronto, Ontaiio. 

Inchidbi artkks on vanous 
of die r ■ irif III ediiiBfcinil 
Sow of Hkc ^mMi^ is in Vi^m^ ^ 
inckiM^ SMT iuues ape flksti^t^ fif 
staB stwbM ani wmmarin of fp^aduatt* 
theses in edncaiian. 

This puMioaiiBn is issued cfuartrrK 



H% onadian EI^m^ ation Ass(K-iation. 
ft^' IS Strtn^t. ^Minvitn 5. Ontario 
Mr-fp^pNons to thr ^ hurst mav be oh 
I V writing t» nn* aAiM < s abov< 



Frederic 
.arolina. 

^i d to 
■ i i i m s in the 
4 indtvidual 



Kuv- Durban 



rwc'iissicm 
oi tK- wasurt 
2) rt 



4 research un the A ineiopmcit" 
mf- of tests and measiMmipnts ir 
industry, and gnvemment 
ions of testing pii a na ms b* 
for variM» p u rposns and 4 
ts ncil» n ef tinmt to t^- 
it field. Wm*k rrviews at tb« 
irf each isaar p r iw wt i ctirrent 
«i# inform afcii for kntber stud\ 
(|uarterK. one cMnplete voi 
par calendar year. For sinfQH- 
write to Educmhamd mml Psych 
Measurement. Box Collefp 
, Durham, Nnrth Q — ttu a. 

Bureau 

lonal ReMvch and Servv^ 
UniveiMty. Caiambus, 

articka cm 
m educaftSMial 

:s that are nc 
areas. 






in schook «n semi mkmA pru^ 



The Hrscarch iJU'raturr 



lems. Empliasis is on techniques used, 
methods emphiyed, and major out- 
c^)mcs. Authors include public school 
personnel, college professors, special- 
ists, and local unit research direc*tors. 

Each magazine issue carries inter- 
(*sting articles describing research, ex- 
perimentation, and other projects in 
Illinois sch(K)ls. Contributions to the 
magazine are welc*ome and sought. 

This journal is published three times 
a year and distributed to members of 
the Illinois ASCD as part of their 
annual membership. Subscriptions 
available at $3 a year. Order from 
Administrative Relations Director, Illi- 
nois Education Association, 100 East 
Edwards Street, Springfield, Illinois. 

Jomal ol Edwfiio— I Rctcvch. Dem- 
bar Publications, Inc. Madison, Wis. 

Reports significant research beiag 
carried on in a wide variety of areai. 
It attempts to be inclusive by statiag 
very clearly the problem, sKthod of 
procedure, sumMry conclusions, mmd 
implications. Each article is a whcilr 
study, not a synopsis or simply a ir- 
view. There are certain issues whicli 
are devoted entirely to one subject, md 
annotated bibhographies are included. 

Published monthly nine times a year. 
Single copies may be obtaiaed by writ- 
ing to Dembar PubUcatiow, Inc., Boa 
1605, Madison, Wisconsin. 

JovMl ol rn il MoiKlsa- 

Dembar Publicatkms, Inc. Madison^ 
Wisconsin. 

This is a periodical le p or t of sci- 
entific investigations relatiog to child 
development, curriculum, learning. 



ti*uching su]XTvisi()n, measurements, 
statistics, and experimental technitjues. 
Whole research projects an* reported 
and a selected bibliography is included 
with each study. 

Published <juarterly in September, 
December, March, and Jum*. Single 
c*opies are available by writing to Dem- 
bar Publications, Inc., Box 1905, Madi- 
son, Wisc^msin. 

NEA Research ■oliih National Edu- 
cation Association. Washinf^on, D.C. 

Research reported is <if a cumulative 
(survey) type. Information on a sub- 
jec*t is accumulated and the figures are 
reported. There is little effort to in- 
terpret data in terms of the conclusions 
and implications involved. Information 
that has been brought together can be 
used, however, to view trends in the 
various areas studied. 

Published in March, May, October, 
and December. Single copies may be 
obtained by writing to Publications- 
Sales SecHon, NEA, 1201 Sixteenth St., 
N,W., Washington, D.C. 

Psychology 

Atmtkm JootmI of Psychology. Uni- 
versity of Texas. Austin, Texas. 

Reports in a sophisticated way re- 
search studies in the field of experi- 
mental psychology. Five departments 
are represented in every issue: articles, 
minor communications, descriptions of 
apparatus, notes and discussions, and 
reviews of books. Studies are reported 
in their entirety and include purpose, 
procedure, subjects, and summary and 
conclusions. 

The American Journal of Psydiology 
is published quarterly, on the 15th day 
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rt mber. Back numbers ami v(»lumcs 
may hv obtained from the Bnsinrss 
Manager. Dipartmcnt of Psycho|np\\ 
Mt'/< s IfalK Uiuv(»r.sity of Texas Aus- 
tin. Texas. 



ogr- British Psycb^logical ,Si<irtv Lun- 
iliwi. England. 

Rt^jorts research in a rigorous tra- 
ditk«ial way. Intmdiic*tHNi. prtKx'titires. 
r(Hi«lls. and diseiHNtion are the ruK* for 
evwn paper. The topics dealt witJi ar*^ 
\enr specific in ntiire and re(|uire a 
sophisticated Iwmlcdge of statistics 
and psychology fnr underst ant ling. This 
publication is issiMnl three timts a vear. 
in Febnian*. June and Novetnber. and 
may be obtainrd by uritn^ to the 
Manager. B.J.E.P DeportMnt, Mc- 
thuen and Co., Ltd,. 30 Emtx Street. 
Strand, London. U .C i Eaj^nd. 
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CUM Stttdy. Child Study Association 
of America. New York, New York. 

Centered on par(*nt (education, so is 
relatively free from the rigorous type of 
scientific research present in other jour- 
nals. The publication deals with a cen- 
tral topic in each issue. Articles are 
written mainly by medical doctors but 
also by parents and other people inter- 
ested in children. 

Published quarterly in the summer, 
fall winter, and spring. Single copies 
are available by writing to the Child 
Study Association of America, 132 East 
74th' Street, New York, New York. 

CUMhood EAmUhNi* Association for 
Childhood Education International 
Washington, D.C. 

Presents articles by qualified people, 
dealing with elementary school age 
youngsters. Although research topics 
are not reported as such, they are 
(|uoted and cited in the articles. Cur- 
rent books and pamphlets for both chil- 
dren and adults are reviewed. 

Published monthly September 
through May. May be obtained in sin- 
gle issues by writing to ChiUkood Edu- 
cation, 3615 Wisconsin Avcmt, N.W., 
Washington, D.C. 



Association for 
Supervision and Curriculum Develop- 
ment, NEA. Washingtan, D.C. 

Centered on a general cunicuhim 
theme with features on specific aspects. 
Authors of the articles represent state 
departments of education, public 
schools, and universities. Incorporated 
into each publication is a section en- 
titled ''Research in Review," which 
summarizes some of the current ex- 



periments and r»^»arch projects he iaff 
carried on in the art*a of the ^general 
theme. 

Copi€»s of Educational leadership are 
published mcmthk. CVtober thnmgh 
May. and may be (obtained by writing; 
to the AsstKiatiim for Supervision and 
Curricuhim Development, NEA. 12M 
Sixteenth St . N.U . U oshingtim, DC. 
DC. 
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Published Msnthk' OctcilK^r through 
May. May hr obtained b*- addressing 
a request for angle copies to the Uni- 
versity of Oiicago Press, 5750 Ellis 
Avenue, Chicaf^, Ilinois. 

ExccfUiM^ OMm. Council for Ex- 
ceptional CWiren, NEA. Washington, 
D.C. 

Composrdrtf: a) eaipirically ground- 
ed researdi lindies frf current interest 
and b) aititles oo various problems 
regarding meptioMl children that re- 
port progresi or stair •pinions. A sec- 
tion of everr issue is dbvoted to review- 
ing new hatdn in thr ttld. 

Issued miflithly, Sii^mber to May. 
Single copies are av^Mbble by writing 
to the Council for Exceptional Chil- 
dren, NEA, 1201 Sixteenth Street, 
NAV., WaAington, D.C. 

IowmI «I EdvaliM. Boston Univer- 
sity, School of Education. Boston, 
Massachusetts. 

Since DecMker NK, this magazine 
has considered a aim^ topic and has 
featured artkiet bjr only one or two 
authors. The tubfccts picked concern 
both elementary and secondary schools. 
Some of the imies aae devoted entirely 
to reporting reseaarii studies, while 
others sin^ state what has been 
learned frooi the literature and research. 

Published quarterly in October. De- 
cember. F^niary. and April. Sin^ 
copies may he obtained by writing to 
Jourmd of Cducatkm, 332 Bay State 
Road, BoilM^ MaaMdiuaettt. 

University ftess. Wathingtvm, D.C. 
The purpose of tbt journal is three- 



fold: first, to stimulate the collection 
and facilitate the dissemination of facts 
about the education of Negroes; second, 
to present discussions involving critical 
appraisals of the proposals and prac- 
tices relating to the education of Ne- 
groes; and third, to stimulate and spon- 
sor investigations of problems incident 
to the education of Negroes. 

Published quarterly in winter, spring, 
summer, and fall. Single numbers are 
available by writing to the Journal of 
Negro Education, Howard University, 
Washington 1, D.C. 

lomal of Teacher Edmlioa. National 
Commission on Teacher Education and 
Professional Standards, NEA. Wash- 
ington, D.C. 

Composed of articles and research 
studies dealing with various aspects of 
teacher education. Some of the research 
topics are reported in detail and include 
conclusions and implications. Other 
studies and articles merely cite research 
done in that area or state an educa- 
tional viewpoint. "With the Research- 
ers,** a regular feature by the research 
editor, draws attention to significant 
problems or findings in the field which 
need recognition. 

Published in March, June, Septem- 
ber, and December. Single issues niay 
be obtained by writing to the Journal 
of Teacher Education, 1201 Sixteenth 
Street. N.W., Washington, D.C. 

The Natioml ElcMenlary Prlacipal. Na- 
tional .Association of Elcincntarx' SchiX)l 
Principals. Arlington, X'irginia. 

Articles in each issue center on a 
single topic in elementary education, 
such as handwriting, grouping, art. etc. 
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Articles are published every year which 
inccjrporate much of the research con- 
cerning the elementary school princi- 
palship. Research is cited in the body 
of articles and in tlie bibliographies 
which are listed at the end of many 
articles. 

Published in September, November, 
January, February, April, and May. 
Single copies may be obtained by writ- 
ing to the Sational Elementary Prin- 
cipal 1201 Sixteenth Street. N.VV., 
Washington, D.C. 

Phi Delta Ktfrtm. Phi Delta Kappa. 
Bl(M)mington, Indiana. 

This is the official journal of Phi 
EX'lta Kappa, professional educational 
fraternity. Research articles, reviews, 
and news alert the professional edu- 
cator and interested layman to signifi- 
cant developments in education. 

The Kappan is published nine times 
a year, from October through Jime. 
Subscriptions may be obtained by writ- 
ing to Phi Delta Kappa, Inc., 8th and 
Union, Bloomington, Indiana. 

School md Society. Society for the 
Advancement of Education. Lancaster, * 



Pennsylvania. 

Research studies are incorporated 
into the periodical on a limited basis. 
All articles are very short, easy to read, 
and run the full range of educational 
topics. 

The magazine, in its fortieth year of 
publication, is published biweekly. 
Single copies may be obtained from 
Business Press, Inc., 10 McGovem Ave- 
nue, Lancaster, Pennsylvania. 

School Review. Department of Educa- 
tion of The University of Chicago. Chi- 
cago, Illinois. 

Composed of a series of articles 
which center on a single topic or gen- 
eral area. While the material presented 
IS not a report of research, it does offer 
some of the best writing in the field on 
the particular issues discussed. Besides 
the articles, tliere are sections devoted 
to book reviews, new books in the field, 
and editorials. 

Published quarterly: spring, summer, 
autumn, and winter. Single issues are 
obtainable by writing to The University 
of Chicago Press, 5750 Ellis Avenue. 
Chicago, Illinois. 



MONOORAPHS 

Materials produced in pamphlet form are convenient to read, file, and 
use. Researcliers can select tlie particular titles which reflect their 
interests and ignore titles wliich seem not to deal with what is wanted. 
This quality represents one large advantage of monographs which books 
and many journals do not share. 

Whot Rceeorch Sojn to the Teocher. (A The purpose of the pamphlets is to 
Series of Monographs). Department of keep the classroom teacher abreast of 
Classroom Teachers and American Edu- current research in the field of educa- 
cational Research Association. NEA. tion. More than thirty-three separate 
Washington, D.C. publications are available on a wide 
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variety of topics. Opinions on various 
studies are included in the reviews. At 
the end of each pamphlet there is a list 
of research references for further study. 

Single copies of the What Research 
Says series may be obtained by address- 
ing communications to the National 
Education Association, 1201 Sixteenth 
Street, N.W., Washington, D.C. 

RtttMch Rnmmt. California Teachers 
Association. Burlingame, California. 

These research bulletins are intended 



for use in California and apply mainly 
to situations in that state. However, 
some of the topics treated are of a more 
general nature and hold significance for 
all. The subjects are of a fairly con- 
crete nature and make for easy reading 
and understanding. 

The publication has no set dates for 
issuance, but single copies will he 
available by title through the Publica- 
tions Supply Department, CTA, 1705 
Murchison Drive, Burlingame, Cah- 
fomia. 



One troublesome problem that an investigator faces is to preserve 
in convenient form the information he has acquired from his reading. Of 
course, his reading will be highly selective and account for only a small 
part of the voluminous amount available. But even this selective part 
will be cumbrous to retain in memory. 

The solution is to devise a suitable system. One flexible approach to 
the keeping of records is that of recording titles, authors, references, 
sources, and notes on index cards, which are then filed according to some 
system. The cards may be filed alphabetically by name of authors. This 
does not help to locate cards dealing with specific subjects, so it may be 
preferable to file cards according to subjects, alphabetically arranged. Or 
each item may be catalogued alphabetically under the name of the school 
system or university where the work was done. In this system, the card 
references from a single research group are kept together, regardless of 
the author. Arrangement by broad subject categories, between which 
there are few overlaps, is usually desirable, whatever system is emploved 
within each category. 

All in all, the major point is that the investigator should keep his 
records and notes in some fashion which will be easy and convenient for 
his work. The individual has to decide how extensive a system he wishes 
to use. Once decisions have been made, even a small amount of effort 
spent in keeping notes will seem justified. 
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Heaemrch far the frmcfUkmer in Education 

1 1 It would br a miiUke to atsume that the reading of thU hook- 
II or any other hook, for that matter-wouM equip educational 
I practitioner! with all of the ideai and skills neceuary to pro- 
ductive research. A hook can he stimulating and informative, 
packed with studied and tested ideas. But it remains, after all. 
an inert thing without life, bkmd. and judgment* There is quite a chasm 
hrtween ideas put in print and the effective use of those ideas in the 
complex and demanding world of reality around us. 

The translation of a theorHiral. scientific approach into warm and 
human educational practices may easily represent tlie most rewarding 
and inventive challenge of all. Tlie administrator and teacher who would 
take the next step of finding ways to convey research ideas and teclmiques 
from a hook to everyday practice need an additional set of suggestions 
which have been tested in actual sitiuitions with on-the-job scliool people. 

SOMI WM d l li U M MSIAKN TIAMINO 

Fortunately, there have been a few attempts to experiment with this 
problem, and the results have been promising. Two of these trials have 
been selected as hMrkground referents for this chapter. 

ffct Hmm Jmmf MnfaraMM. The first experiment was comprised 
of a series of ten two*day research training conferences conducted be- 
tween April 1953 and May 1954 for sixteen public school people from 
six New Jrrse) public school systems. The cotiferencrs were sponsored hv 
the Horace Mann-Uncoln Institute of Sclmol Expt^rimentatkm at Teacliers 
College. Cokimhia University, and resulted in u |Niblication which pre- 
sented detailed descriptions and evaluations of tlie procedures empbved.^'' 
These conferences were primarily c^mcemed with human relations pr«>b- 
Irms such as interactions hetwc^ni amsiilUnts and participants and be- 
tween administrators and teacliers. Relatively little stri^ was placed upon 
technical Iramings such as research design and statistical analysis of data. 

Ifct Mblilt MMlMft, T\w second experiment cxmsisted of a series 
of thrre three^y research training seminars lietween Novemlier 19SH 
Mid February 1990. Mure tlian 2*) superintendents, principals, super- 
visors, teachers, and school board memliers from public scInioI systems in 
llhnois participated in the thrre seminars. Consuhaiit teams wvtv com- 
posed of college and public school research specialists. Tlie seminars 
were s ponso ml by the Illinois Curriculum Program, a branch of the Office 
of the iNiMis Superintendent of Public Instruction. Tlie Curriculum Pro- 
fram was thm housed at the University of Illinois. Tliis experiment 
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resulted in a doctoral dissiTtation re|)ort which prejicnted extensive de- 
scriptions and evahiations of plans, results, and instnnmMits eniploviHl.^^ 
Also inchided was a long-time evaluation of what had hap|)ened to the 
partici|)ants, as far as research was concerned, one year later. The semi- 
nars gave emphasis to problems in tin* technology of scImm)I research— prob- 
lem identification, hy|X)thesis making, research design, .statistical analysis, 
and tlu' reiK>rting of results— as well as to difficulties related to interactions 
Ix'tween consultants and learners. 

Because tlu motives and envinmment for ctmferences and seminars 
away from lM>me seemed difl^erent from what might lie ex|>ected in more* 
familiar settings, the Illinois Curriculum Program staff meml>ers conducted 
voliinttM^r follow-np research seminars for approximately forty administra- 
tm and teacliers in two Illinois scliool systems. The seminar sessions 
were Iwld oiR»e each month over one scliool year. Results were com- 
|iarable to tlMm> of tlie original si^minars, and in conclusion a booklet was 
puhlislied which contained tlie reports of several statistical studies con- 
ductinl in local schools of tlie two fyitems.'** 

nMHNma km local msiaicn tramino 

Altliough much remains to be learned concerning the in-scbool re- 
search training of adminiitratori and teacliers, sufficient guidelines, tech- 
niques, and innovations have been demonstrated and evaluated to offer 
valid patterns for individual school systems and individual schools to 
adopt and modify for their own use. This chapter foctises on the need 
for help from outside sources in staiF training; points to wliere such help 
might be secured; describes the nature of efficient training programs; 
emphasizes the intellectual climate of research inquiry; and makes a pro- 
posal for long-range studies in American schools. 

iMp fraiM "MtaM^." The individual school probably comes closest 
to being the laboratory for learning, and it is here that theories can be 
tested against actuality and research results disseminated to others. The 
individual school can exercise more direct control over problems, subjects, 
and materials than can more distant institutions such as state, county, or 
even system*wide organiasations. And the individual school is many times 
cloaer to the learning laboratory idea than are the national projects which 
are currently conducting curriculum studies and making recommendations. 

But in spite of the individual school s advantaged position, sustained 
help is needed if staff members are to develop learnings and skills related 
to the technology of experimentatkm. Probably few elementary schoob 
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(and high schools, too, for that matter) are staffed witli a principal and 
teachers wIk> are already equipped to undertake inquir)', experimentation, 
and research. The necessary attitudes, knowledge, and skills can be de- 
veloped as a regular part of the job through a deliberate program of 
research training seminars which take priority over other in-service tasks 
for a scliool year or longer. Training sessions should be held as frequently 
as feasible, and certainK- more often than once a month. 

Of course, nobody would expect an individual school $taff to start 
from scratch in assembling and organizing all of tlie materials and processes 
desired for a sustained series of seminars. But an outside expert or experts 
might be obtained to work with the staff and with the principal at the 
same time. The wlK>le process of committing a staff to research learnings 
becomes a kind of deliberately introduced and wholesome Hawthorne 
effect, followed by liearty reinforcement, at least during tlie period of 
training. 

The principal sliould participate actively and directly in research 
processes. To become an expert leader in the research function, he will 
perfect his skill when he has undergone some systematic instruction with 
his teachers, has engaged in research study, and has learned to apply his 
knowledge in research operations. 

Smtcm The first source of bdp may consist of the periodical 

literature which reports complete educational research projects. The pre- 
ceding chapter was given over to the identification of these sources. A 
school might profitably subscribe to magazines such as The iournal of 
Educaiiofml Re9earch and the American Educational Research Journal for 
examples of current research topics and procedures. 

The second source includes the beating and obtaining of consulUnts 
to help in staff training and subsequent research activities. ConsultanU 
may be found in colleges and universities, in adjacent school systems, in 
state oflBces of education, and sometimes among the psychok^gical and 
guidance personnel of ones own school system. Larger school systems 
such as New York, Chicago, Philadelphia, Pittoburgh, and Los Angeles 
have kmg employed full-time directors of research and staff members 
attached to the director's office. Within the past few years, new positions 
designated as ^'director of research" have been appearing in many smaller 
school systems located in cities of about 100,000 population. The central 
office in a large enough school system can assist in the provision of con- 
sultant help and training and sometimes can cooperate with adjacent 
school systems and schools. 

Ragardleti of the louroe for a comulUnt, it it crucial to obtain the 
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right individual for jobs of this sort. There have been instances where 
a statistical researcher, competent in his own right, inadvertently quenched 
more fire than he fanned among naturally insecure beginners. This can 
be disastrous. The Illinois Curriculum Program looked for and found 
consultants who met three basic criteria. J\e criteria called for consultants 
who were: 

1. Experienced educators. knowledr«*iUe with respect to both theory 
and practice in schools 

2. Reputable producers of research iuit might be expected to con- 
tribute to improved educational practice 

3. Skillful and sensitive persons in the areas of human relations and 
group climate. 

Individual scIkx>I groups owe such careful pre-thought both to them- 
selves and to the potential consultants. The consultant should be ac- 
quainted with what is expected from him and what is needed by the par- 
ticular principal and scIkioI staff wlien he is invited. Criteria to aid in 
finding the kind of person needed should be worked out well in advance 
if the schools objectives are to be served. 

We cannot leave the matter of subscribing to research journals and 
securing consultants witliout considering the necessity of having access to 
funds budgeted for research and devekipment. Expenditures for these 
purposes represent legitimate uses of monies formerly taken for granted in 
scIkx>1 activities such as institutes, workshops, lectures, and the like. As 
school systems become accustomed, as has American industrv. to improving 
their product through research, regular expenditures for this purpose will 
lie regarded as quite ordinary. 

Ih9 notuw 9t traMfif prafrmm. In the past, school administrators 
and siiper\ isors liave tended to use their own graduate courses as the model 
for staff in-ser\ice programs. Thus, lectures and speeches by specialists, 
readings folbwed b\ discussions, and demonstration teaching programs 
have composed tlie usual range of technicpies empbyed to bring informa- 
tion to scliool staffs. In other words, tlie techniques for in-service education 
activitic^s Imve clos4>l\ followed tlie traditional didactic college pattern of 
c*ourse offerings and direct teaching of "coune** content. 

But direct teaching of this sort has lx?en recognized as ciirtibrous and 
inefficient for S4)me types of learnings. The first obstacle introduced by 
college style instruction has to do with tlie amount of time required for 
learnings to mature. The second obstacle encountered in this essentially 
verbalistic approach to learning concerns tlie failure to translate course 
content into imnu^iato lieliavior and action. 
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During World War II, the miHftary forces drveloped a worlnMe and 
efficient kind of training devior i^Mch greatly speeds the time wt^fmed 
for new learnings to take hoM md which slHiB instruction dkm to Hie 
level of desired behaviors. ThmJbaam is kii— ■ riwmtoton. It Ms been 



employed succeirfully in traiM^ipBgrains W hwiess and iMkastry and 
in certain experianental coUeprtHMnla wh«r> Muivioral types of learn- 
ing are desired. 

The Illinois C^irricukim l^^mm ^velopiAaBdl tailed »4ind of simu- 



lated training that was uasd atfAwM^ide for lli ■■■ iiii li laning seMMrs 

which were compressed into Wmm^^m time spaH. A traiM^g device from 



the armed farces offered leadawriAt seminars. Diving the war, the serv- 
ices operated a so-called dry-fwfcr Gaining mtttay men in battle cona- 
tions, whnrc the dangers of gM^B tkr kaaards of real shooling 
were eliBinated. Adapting Ais iAia. a '^dry-mmT imining device in ve- 
search pBctices was devebpedl wkiHi ppovicUi m HMing for p ract i ce and 
skill dewfepment in a thrial fcni *mi psych^gicdly safe envinmment 
where nHtakes are nontoxic, lifr ^tovice strand whole penan learning 
where participants could feel, art mil^tmk tke-lsHaiiig experience rather 
than just listen or talk abnnt wlM nMs lannring. Small ^mips of 
teacliers and administralM. nririi aMnllHt tnav, were organized into 
research groups to perfoms ^UriMM^Me** wmmmk profects on the spot. 
The participants used each ^Ikm m "sanifibi" and empk>yed synthetic, 
hut real enough, research psnUnBs which iiitgriBlaJ them. They identified 
a problem, stated hypotheses, chose a research design which fitted their 
problem, selected a statistical test, collected the data, statistically analyzed 
them, came to decisions, and received immediate feed-back in terms of 
group critiques. 

All of this was done in a "not-playing-for-keeps** atmosphere where 
freedom to admit ones own lack, and then learn, were prized. In the 
beginning stages of new, unfamiliar, and sometimes threatening learning 
episodes, a "dry^run** experience not only permits but also encourages 
uninhibited learning and skill development. Anyone participating in a 
simulated learning situatk>n knows that he is only practicing and that real 
people and real events will be neither harmed by his ignorance nor lielped 
through his genius. Later, after confidence and courage have been built 
through demonstrated successes and generous reinforcement from the 
trainer (consultant), a ratlier rapid transition can be made to real research 
problems involving real p't^/k in real bnming situations. This brief 
description of training Itammfb simulation aantains many implications tor 
the sort of consultant naaiM^ pHncipab and teachers. 

m 
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Nurturifif IM mtaNsctuol cliinof% of rM#orch mquiry. Repeatedlx 
tlirougliout tliis Ixwk, we liave adopted the bias tliat rost-arcli represents 
an intensely personal commitment; it must Ix-long to the researcher. Thus, 
the independent questions and doubts of practitioners, wIk) are the pre- 
sumed "consumers" of practices recommended for school use, form one 
(juite necessar) origin for studies designed to check and demonstrate the 
worth at home of popularized and gnierali/ed innovations. It might be said 
that practices hitclied to recently nAremed approaches to teaching-such 
as phonics, modem mathematics, pn^ams for the gifted, ability grouping, 
televised instniction, enrichnM^, science fairs and competition, language 
laboratories, teaching machines, departmentalization for elementary schools, 
et cetera, et cetera-should not he precipitously adopted in the absence of 
many original research studies, ami replications of research studies, per- 
formed in the individual schools contemplating revised teaching. 

Of course, popular and relatively recent innovations like those men- 
tioned in the preceding paragraph represent only one source for needed 
studies. They are listed because of their currency and implied insistence 
and because they confront schools with pressing and immediate problems. 
They are tiot listed to imply a resistance to innovations in general or to 
suggest a retreat to the status quo. Innovations call for different kinds of 
thinking and are irritating enough to be generally beneficial But if their 
effects are sometimes good, they may become powerful when selectively 
tested and followed through into practice on the basis of local research 
triab. On the other hand, the habit of conducting local pre-tests and 
research trials before committing a school to innovations, qua innovations, 
undoubtedly would result in throwing out a proportion of the "new** prac- 
tices on the educational horizon. 

The usual and customary practices which have become part of the 
baggage carried abng by any school staff form another necessary point 
of origin for studies designed to check what may be too comfortably 
accepted and to help invent fresh "bright ideas" which have not occurred 
to others. In their own private woHds, many principals and teachers may 
have k)ng "known" better ways to teach almost anything to almost anyone. 
Their secret knowledge may have been kept buried (and probably was) 
becf Lso they lacked acceptable ways to devek>p and verify it. With the 
gaining of knowledge and skill in research and devebpment, this private 
world of truly experimental research can be widely shared with colleagues 
and others interested in public schools. 

It is tempting in a book such as this to orpnize and list groups of 
potential studies deemed to be beneficial for someone to pursue somewhere. 
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But this temflHii unm dcB^te% tadf tiniiii good iliiiiii begin 
gOQil problow mi good r| n rifh — whisli hwe been idntfied by nd». 
vkU reseaKben who care eiiMgk «e them thwmgpi the rigov «f 

careM resevck PnctitioMti cm be iH^Bi to learn and aequire iv 
and ddlls in adaik^n to the methods Mdiirfbiiques for MB^cting ; 
s t iadi ii , but aalaaaing what is worik mmmm^gfUt^ tmm m asdless 
of possible dkmmm awl deciding upM Maiive wa»» to Msatigate Kt^m m 
quito persoMl mmm that really csMat W done \m m «aider. I* 
this pocess ol aalaaliag aad decidii^ i 
tual dimato w&km widch research inquiry 

proceu itaaV is aaaanMally creative, 
educaUonai fob to^ done through 
aknost ov^qpowaaiB^. This imbalance Luto iiin it^'mmti and the func* 
tions of paaple in research becomes a amjor pasMav on the national 
scene. To Mni fi ai i the large dimensions of impmmm^ aincational practice 
in American schoais, we can look at some aalactad OTMI statistics which 
indicate the nutgnitude of the enterpriae. 

# In 1963S4 *m Anmtca there toate ^^nmi S2J090 fmkHc eUmeniary 
Mchoek mnd 25J0im pnMkr mcondary wdmak. 

# Enrolled in Ifcaar eehoola were dfJ mittdn (imareaeing at 
the rate of 1 iii W i i pnpiir per year fortheleetU yeen). 

# Theee 41.8 pnpds were iet^ by 900jm temehen in the 
fmbUc elementary eckeoU and SSOfiOO tondbart in tha pubHc eecondary 
Mchoole, far a total of IJSSOflOO teachen. 

# Expendituree for thk enterpriee totaled $17 bUhan, making pnMc 
education one of Amaricns greateet bueimemee.^^ 

In the past n> manage and linpiu>ii this 1)ig business,'' we have 
improvised with minimal expenditusns for research and development of 
any sort and only a tiny hamtfu! of panple who piniaaaarl to be r cs eai cl i eis . 
And these people have been located amatly in cnl^g^i and universities- 
not in the pnbUc schoob where learning takes plaoa. 

This stoto of affairs has put a stanqp on educnHanal research and has 
given it certain general characteristics. Edncattonal research studies tend 
to be fragnaantary and discontinuous. They tend, iar the want of resaarch 
peraonnd, to be performed once and never again. Their empirical taata qr* 
kieas tend to be boated in one school or school syatom, in one part of the 
nation, to be assumed as valid for schools or dassraoms averyw h ere. 

The taak ia shaaily great In siae and daring. To meet tha chaDange, 
we had best diink In terms of m amic e on-going research studies in nu- 



■KTOus situatiM acM «r couiiirv , i nn t xi In iidi||^nous local funds. 
WkM wc thnk of jkmik ^-^nitiMl praUnns iMtg aMacked simul- 
^"■ ■■■ " ■ 'y ^» t twi n wii mk^^\ iiiiii iwatail ^ parts o* the countn% 
taikrr than in me ^y^^gm i Jya rrn t to * » iMvrrsitw we can get 

ji feeling for what revaavi k piinteA^ can do. 

To accc p jfl i a ni ^ n^ like *e »hift w««cf i» Mfgeealed two 
'•■iippw in our ikmimm^ m «rarc4i seem obvmii The first ccuKems the 
*^ '■•^M^ HHwfcer of peopk v ho engage in research. 

HflM»iy, the pnal «f more tlian me arid one- bait nnilion administrators 
mmk tandiers in Amrican toliools k potentially rich and inviting. These 
mt the people in^mnd in n ni nr a l laboratories for tnlncaAinnal experi- 
iMitelion. If we hnnr the mit to mnlch them wilk the atflage mmk mmer- 
ms_ maarch i r i iii*^!, ^ hmv tlie knowMpr hot Inck tkr mmbers 
the mamfmmm problem cmaiie raaaftved. The second 
ibanciag of — u r ch and d g^we topma u t in local school 
When tegular schod funds can he ammaM) tiiijpti i! 

he scrimsly evalwd. Reaemch ^nts 

leem to cnasr a kmiiing influencr an Ae reienrch 
^ ^ UnfortiM^fci|^uiiie gi nnli i generaly insiallini certain 

^ ' ^ ^ ^^ ^ highly g ■ tf illi and local n^ 

ilued. ImplWiy, thia aait «f bias rules out nnsnerous pro^s that 
Inenl people would ha eager «• pursue. 

When hical schnnl people can gain recognition plus additional financial 
remuneration for lhair conMbutions to the science of education, then a 
motive force may he put in motion which wil cnrrv itself. Vltmm school 




practitioners in la^pr enou^ numbers msi jnin as equals the snmnchers 
m eAmtion and nanduct llmir dos e to hom e type of experiaaaatation. 
when financial support iar reseatdi permits praeNtioners to do the 



of studies that they asnliy want lo do in many schoda, then wc 
may be in that favmable pmMon to devebp a maasive attack on our best 
problems. 
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